Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.05,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2200,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 2200,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.478706
Epoch 1
Validation binary_cross_entropy = 0.289372
Epoch 2
Validation binary_cross_entropy = 1.378152
Epoch 3
Validation binary_cross_entropy = 1.036968
Epoch 4
Loss = 7.0431e-01, PNorm = 96.4336, GNorm = 5.5253, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.295685
Epoch 5
Validation binary_cross_entropy = 0.746380
Epoch 6
Validation binary_cross_entropy = 1.587386
Epoch 7
Validation binary_cross_entropy = 0.368065
Epoch 8
Validation binary_cross_entropy = 0.316209
Epoch 9
Loss = 4.6987e-01, PNorm = 96.7672, GNorm = 1.4086, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.779911
Epoch 10
Validation binary_cross_entropy = 0.535561
Epoch 11
Validation binary_cross_entropy = 0.295477
Epoch 12
Validation binary_cross_entropy = 0.487579
Epoch 13
Validation binary_cross_entropy = 0.410937
Epoch 14
Loss = 3.9508e-01, PNorm = 97.0928, GNorm = 1.9290, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.326036
Epoch 15
Validation binary_cross_entropy = 0.397974
Epoch 16
Validation binary_cross_entropy = 0.376024
Epoch 17
Validation binary_cross_entropy = 0.408262
Epoch 18
Validation binary_cross_entropy = 0.638505
Epoch 19
Loss = 2.8291e-01, PNorm = 97.3326, GNorm = 4.3847, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.389358
Epoch 20
Validation binary_cross_entropy = 0.461680
Epoch 21
Validation binary_cross_entropy = 0.341077
Epoch 22
Validation binary_cross_entropy = 0.464555
Epoch 23
Validation binary_cross_entropy = 0.414205
Epoch 24
Loss = 1.9608e-01, PNorm = 97.5482, GNorm = 4.8816, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.364495
Epoch 25
Validation binary_cross_entropy = 0.662764
Epoch 26
Validation binary_cross_entropy = 0.382537
Epoch 27
Validation binary_cross_entropy = 0.425573
Epoch 28
Validation binary_cross_entropy = 0.387907
Epoch 29
Loss = 1.6239e-01, PNorm = 97.7531, GNorm = 4.2389, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.368348
Epoch 30
Validation binary_cross_entropy = 0.488063
Epoch 31
Validation binary_cross_entropy = 0.349186
Epoch 32
Validation binary_cross_entropy = 0.392091
Epoch 33
Validation binary_cross_entropy = 0.524529
Epoch 34
Loss = 1.7031e-01, PNorm = 97.9868, GNorm = 2.0641, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.439629
Epoch 35
Validation binary_cross_entropy = 0.436834
Epoch 36
Validation binary_cross_entropy = 0.412872
Epoch 37
Validation binary_cross_entropy = 0.386531
Epoch 38
Validation binary_cross_entropy = 0.394835
Epoch 39
Loss = 1.4433e-01, PNorm = 98.2210, GNorm = 2.7090, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.437525
Epoch 40
Validation binary_cross_entropy = 0.386860
Epoch 41
Validation binary_cross_entropy = 0.424563
Epoch 42
Validation binary_cross_entropy = 0.452693
Epoch 43
Validation binary_cross_entropy = 0.456470
Epoch 44
Loss = 1.7553e-01, PNorm = 98.4257, GNorm = 2.5949, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.474598
Epoch 45
Validation binary_cross_entropy = 0.406002
Epoch 46
Validation binary_cross_entropy = 0.385032
Epoch 47
Validation binary_cross_entropy = 0.407066
Epoch 48
Validation binary_cross_entropy = 0.437865
Epoch 49
Loss = 9.7110e-02, PNorm = 98.6443, GNorm = 1.7936, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.418555
Epoch 50
Validation binary_cross_entropy = 0.380253
Epoch 51
Validation binary_cross_entropy = 0.385965
Epoch 52
Validation binary_cross_entropy = 0.437439
Epoch 53
Validation binary_cross_entropy = 0.453426
Epoch 54
Loss = 1.3558e-01, PNorm = 98.8737, GNorm = 2.2319, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.429221
Epoch 55
Validation binary_cross_entropy = 0.438903
Epoch 56
Validation binary_cross_entropy = 0.435919
Epoch 57
Validation binary_cross_entropy = 0.520456
Epoch 58
Validation binary_cross_entropy = 0.454612
Epoch 59
Loss = 9.7203e-02, PNorm = 99.1091, GNorm = 2.4945, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.545247
Epoch 60
Validation binary_cross_entropy = 0.572022
Epoch 61
Validation binary_cross_entropy = 0.605036
Epoch 62
Validation binary_cross_entropy = 0.617116
Epoch 63
Validation binary_cross_entropy = 0.609348
Epoch 64
Loss = 6.7576e-02, PNorm = 99.3549, GNorm = 5.1857, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.568373
Epoch 65
Validation binary_cross_entropy = 0.585120
Epoch 66
Validation binary_cross_entropy = 0.568226
Epoch 67
Validation binary_cross_entropy = 0.561180
Epoch 68
Validation binary_cross_entropy = 0.476685
Epoch 69
Loss = 8.9185e-02, PNorm = 99.5666, GNorm = 2.3679, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.509483
Epoch 70
Validation binary_cross_entropy = 0.550314
Epoch 71
Validation binary_cross_entropy = 0.484870
Epoch 72
Validation binary_cross_entropy = 0.492974
Epoch 73
Validation binary_cross_entropy = 0.521534
Epoch 74
Loss = 3.9369e-02, PNorm = 99.7975, GNorm = 1.2708, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.552759
Epoch 75
Validation binary_cross_entropy = 0.573972
Epoch 76
Validation binary_cross_entropy = 0.591435
Epoch 77
Validation binary_cross_entropy = 0.577839
Epoch 78
Validation binary_cross_entropy = 0.554588
Epoch 79
Loss = 7.1904e-02, PNorm = 100.0086, GNorm = 2.3071, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.587046
Epoch 80
Validation binary_cross_entropy = 0.538623
Epoch 81
Validation binary_cross_entropy = 0.514790
Epoch 82
Validation binary_cross_entropy = 0.515361
Epoch 83
Validation binary_cross_entropy = 0.522959
Epoch 84
Loss = 4.5582e-02, PNorm = 100.2040, GNorm = 2.0749, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.567633
Epoch 85
Validation binary_cross_entropy = 0.504026
Epoch 86
Validation binary_cross_entropy = 0.544686
Epoch 87
Validation binary_cross_entropy = 0.644679
Epoch 88
Validation binary_cross_entropy = 0.556859
Epoch 89
Loss = 5.4290e-02, PNorm = 100.3761, GNorm = 1.7809, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.560898
Epoch 90
Validation binary_cross_entropy = 0.598126
Epoch 91
Validation binary_cross_entropy = 0.656695
Epoch 92
Validation binary_cross_entropy = 0.580341
Epoch 93
Validation binary_cross_entropy = 0.563671
Epoch 94
Loss = 7.7657e-02, PNorm = 100.6189, GNorm = 3.0928, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.574373
Epoch 95
Validation binary_cross_entropy = 0.575476
Epoch 96
Validation binary_cross_entropy = 0.634511
Epoch 97
Validation binary_cross_entropy = 0.603837
Epoch 98
Validation binary_cross_entropy = 0.584021
Epoch 99
Loss = 7.3165e-02, PNorm = 100.8302, GNorm = 1.0439, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.587295
Epoch 100
Validation binary_cross_entropy = 0.641143
Epoch 101
Validation binary_cross_entropy = 0.677933
Epoch 102
Validation binary_cross_entropy = 0.620403
Epoch 103
Validation binary_cross_entropy = 0.589624
Epoch 104
Loss = 3.9056e-02, PNorm = 101.0344, GNorm = 1.0051, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.606997
Epoch 105
Validation binary_cross_entropy = 0.623575
Epoch 106
Validation binary_cross_entropy = 0.619240
Epoch 107
Validation binary_cross_entropy = 0.656652
Epoch 108
Validation binary_cross_entropy = 0.646232
Epoch 109
Loss = 7.2030e-02, PNorm = 101.2194, GNorm = 3.4622, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.682647
Epoch 110
Validation binary_cross_entropy = 0.641180
Epoch 111
Validation binary_cross_entropy = 0.629141
Epoch 112
Validation binary_cross_entropy = 0.642434
Epoch 113
Validation binary_cross_entropy = 0.742937
Epoch 114
Loss = 6.3660e-02, PNorm = 101.3979, GNorm = 3.3536, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.831635
Epoch 115
Validation binary_cross_entropy = 0.666535
Epoch 116
Validation binary_cross_entropy = 0.672172
Epoch 117
Validation binary_cross_entropy = 0.670547
Epoch 118
Validation binary_cross_entropy = 0.659987
Epoch 119
Loss = 6.7206e-02, PNorm = 101.6295, GNorm = 0.8787, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.663083
Epoch 120
Validation binary_cross_entropy = 0.633512
Epoch 121
Validation binary_cross_entropy = 0.625133
Epoch 122
Validation binary_cross_entropy = 0.624009
Epoch 123
Validation binary_cross_entropy = 0.614750
Epoch 124
Loss = 5.5074e-02, PNorm = 101.8399, GNorm = 0.5373, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.705681
Epoch 125
Validation binary_cross_entropy = 0.723993
Epoch 126
Validation binary_cross_entropy = 0.693551
Epoch 127
Validation binary_cross_entropy = 0.712193
Epoch 128
Validation binary_cross_entropy = 0.752896
Epoch 129
Loss = 1.5300e-02, PNorm = 102.0505, GNorm = 0.4090, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.793277
Epoch 130
Validation binary_cross_entropy = 0.709895
Epoch 131
Validation binary_cross_entropy = 0.679953
Epoch 132
Validation binary_cross_entropy = 0.676888
Epoch 133
Validation binary_cross_entropy = 0.786912
Epoch 134
Loss = 7.1452e-02, PNorm = 102.2541, GNorm = 5.9224, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.715006
Epoch 135
Validation binary_cross_entropy = 0.631278
Epoch 136
Validation binary_cross_entropy = 0.648372
Epoch 137
Validation binary_cross_entropy = 0.622706
Epoch 138
Validation binary_cross_entropy = 0.701745
Epoch 139
Loss = 2.7072e-02, PNorm = 102.4710, GNorm = 1.9134, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.756231
Epoch 140
Validation binary_cross_entropy = 0.715419
Epoch 141
Validation binary_cross_entropy = 0.672527
Epoch 142
Validation binary_cross_entropy = 0.660997
Epoch 143
Validation binary_cross_entropy = 0.686970
Epoch 144
Loss = 7.0249e-03, PNorm = 102.6413, GNorm = 0.5487, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.732517
Epoch 145
Validation binary_cross_entropy = 0.721027
Epoch 146
Validation binary_cross_entropy = 0.682647
Epoch 147
Validation binary_cross_entropy = 0.671110
Epoch 148
Validation binary_cross_entropy = 0.677665
Epoch 149
Loss = 6.0585e-03, PNorm = 102.7599, GNorm = 0.9855, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.683142
Epoch 150
Validation binary_cross_entropy = 0.723819
Epoch 151
Validation binary_cross_entropy = 0.778805
Epoch 152
Validation binary_cross_entropy = 0.803160
Epoch 153
Validation binary_cross_entropy = 0.753500
Epoch 154
Loss = 1.5020e-02, PNorm = 102.8377, GNorm = 1.9095, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.753967
Epoch 155
Validation binary_cross_entropy = 0.792710
Epoch 156
Validation binary_cross_entropy = 0.822016
Epoch 157
Validation binary_cross_entropy = 0.831297
Epoch 158
Validation binary_cross_entropy = 0.824534
Epoch 159
Loss = 2.4765e-03, PNorm = 102.9074, GNorm = 0.3324, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.811958
Epoch 160
Validation binary_cross_entropy = 0.817469
Epoch 161
Validation binary_cross_entropy = 0.822500
Epoch 162
Validation binary_cross_entropy = 0.843179
Epoch 163
Validation binary_cross_entropy = 0.890879
Epoch 164
Loss = 1.2439e-02, PNorm = 102.9549, GNorm = 1.1607, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.923972
Epoch 165
Validation binary_cross_entropy = 0.876529
Epoch 166
Validation binary_cross_entropy = 0.788089
Epoch 167
Validation binary_cross_entropy = 0.789932
Epoch 168
Validation binary_cross_entropy = 0.820197
Epoch 169
Loss = 1.1049e-03, PNorm = 103.0168, GNorm = 0.0290, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.849970
Epoch 170
Validation binary_cross_entropy = 0.864633
Epoch 171
Validation binary_cross_entropy = 0.838587
Epoch 172
Validation binary_cross_entropy = 0.848095
Epoch 173
Validation binary_cross_entropy = 0.913817
Epoch 174
Loss = 1.4593e-02, PNorm = 103.1279, GNorm = 0.1481, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.890678
Epoch 175
Validation binary_cross_entropy = 0.866490
Epoch 176
Validation binary_cross_entropy = 0.931001
Epoch 177
Validation binary_cross_entropy = 1.178907
Epoch 178
Validation binary_cross_entropy = 0.829297
Epoch 179
Loss = 2.1673e-02, PNorm = 103.3076, GNorm = 2.8871, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.925941
Epoch 180
Validation binary_cross_entropy = 0.841321
Epoch 181
Validation binary_cross_entropy = 0.886384
Epoch 182
Validation binary_cross_entropy = 0.861828
Epoch 183
Validation binary_cross_entropy = 0.847374
Epoch 184
Loss = 8.8312e-02, PNorm = 103.6225, GNorm = 1.9996, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.833512
Epoch 185
Validation binary_cross_entropy = 0.837030
Epoch 186
Validation binary_cross_entropy = 0.929307
Epoch 187
Validation binary_cross_entropy = 0.835145
Epoch 188
Validation binary_cross_entropy = 0.693542
Epoch 189
Loss = 1.6436e-02, PNorm = 103.8915, GNorm = 0.8790, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.659628
Epoch 190
Validation binary_cross_entropy = 0.671535
Epoch 191
Validation binary_cross_entropy = 0.850402
Epoch 192
Validation binary_cross_entropy = 0.909110
Epoch 193
Validation binary_cross_entropy = 0.739440
Epoch 194
Loss = 2.3659e-02, PNorm = 104.1477, GNorm = 0.5297, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.712305
Epoch 195
Validation binary_cross_entropy = 0.732846
Epoch 196
Validation binary_cross_entropy = 0.766295
Epoch 197
Validation binary_cross_entropy = 0.835978
Epoch 198
Validation binary_cross_entropy = 0.862744
Epoch 199
Loss = 2.6534e-02, PNorm = 104.3418, GNorm = 0.3566, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.766652
Epoch 200
Validation binary_cross_entropy = 0.729952
Epoch 201
Validation binary_cross_entropy = 0.753363
Epoch 202
Validation binary_cross_entropy = 0.757927
Epoch 203
Validation binary_cross_entropy = 0.800984
Epoch 204
Loss = 1.3138e-02, PNorm = 104.4846, GNorm = 1.5761, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.859824
Epoch 205
Validation binary_cross_entropy = 0.938209
Epoch 206
Validation binary_cross_entropy = 0.949569
Epoch 207
Validation binary_cross_entropy = 0.943051
Epoch 208
Validation binary_cross_entropy = 0.951091
Epoch 209
Loss = 5.7568e-03, PNorm = 104.5868, GNorm = 0.4407, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.974842
Epoch 210
Validation binary_cross_entropy = 1.004342
Epoch 211
Validation binary_cross_entropy = 0.988571
Epoch 212
Validation binary_cross_entropy = 0.955653
Epoch 213
Validation binary_cross_entropy = 0.959130
Epoch 214
Loss = 1.8490e-02, PNorm = 104.7213, GNorm = 1.3874, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.934565
Epoch 215
Validation binary_cross_entropy = 0.892713
Epoch 216
Validation binary_cross_entropy = 0.873213
Epoch 217
Validation binary_cross_entropy = 0.860403
Epoch 218
Validation binary_cross_entropy = 0.852996
Epoch 219
Loss = 1.5044e-02, PNorm = 104.8267, GNorm = 0.0429, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.837468
Epoch 220
Validation binary_cross_entropy = 0.846831
Epoch 221
Validation binary_cross_entropy = 0.871979
Epoch 222
Validation binary_cross_entropy = 0.908087
Epoch 223
Validation binary_cross_entropy = 0.912614
Epoch 224
Loss = 1.3985e-03, PNorm = 104.9194, GNorm = 0.0425, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.892914
Epoch 225
Validation binary_cross_entropy = 0.885867
Epoch 226
Validation binary_cross_entropy = 0.885131
Epoch 227
Validation binary_cross_entropy = 0.894830
Epoch 228
Validation binary_cross_entropy = 0.904385
Epoch 229
Loss = 1.1489e-03, PNorm = 104.9773, GNorm = 0.0757, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.909800
Epoch 230
Validation binary_cross_entropy = 0.912437
Epoch 231
Validation binary_cross_entropy = 0.910767
Epoch 232
Validation binary_cross_entropy = 0.846301
Epoch 233
Validation binary_cross_entropy = 0.813692
Epoch 234
Loss = 3.5341e-03, PNorm = 105.0290, GNorm = 0.1494, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.821532
Epoch 235
Validation binary_cross_entropy = 0.814227
Epoch 236
Validation binary_cross_entropy = 0.859454
Epoch 237
Validation binary_cross_entropy = 0.946540
Epoch 238
Validation binary_cross_entropy = 0.939157
Epoch 239
Loss = 6.6156e-03, PNorm = 105.1366, GNorm = 0.1907, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.875267
Epoch 240
Validation binary_cross_entropy = 0.832581
Epoch 241
Validation binary_cross_entropy = 0.829913
Epoch 242
Validation binary_cross_entropy = 0.847851
Epoch 243
Validation binary_cross_entropy = 0.855354
Epoch 244
Loss = 4.4408e-03, PNorm = 105.2317, GNorm = 0.0449, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.845620
Epoch 245
Validation binary_cross_entropy = 0.839536
Epoch 246
Validation binary_cross_entropy = 0.842227
Epoch 247
Validation binary_cross_entropy = 0.862635
Epoch 248
Validation binary_cross_entropy = 0.886533
Epoch 249
Loss = 7.0686e-04, PNorm = 105.2852, GNorm = 0.0474, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.908246
Epoch 250
Validation binary_cross_entropy = 0.926930
Epoch 251
Validation binary_cross_entropy = 0.939896
Epoch 252
Validation binary_cross_entropy = 0.948008
Epoch 253
Validation binary_cross_entropy = 0.952042
Epoch 254
Loss = 8.8721e-04, PNorm = 105.3223, GNorm = 0.0425, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.950795
Epoch 255
Validation binary_cross_entropy = 0.947171
Epoch 256
Validation binary_cross_entropy = 0.942178
Epoch 257
Validation binary_cross_entropy = 0.939364
Epoch 258
Validation binary_cross_entropy = 0.941181
Epoch 259
Loss = 1.0689e-02, PNorm = 105.3467, GNorm = 0.0464, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.979923
Epoch 260
Validation binary_cross_entropy = 1.012208
Epoch 261
Validation binary_cross_entropy = 0.988682
Epoch 262
Validation binary_cross_entropy = 0.968103
Epoch 263
Validation binary_cross_entropy = 0.966700
Epoch 264
Loss = 1.2884e-03, PNorm = 105.3738, GNorm = 0.0339, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.977126
Epoch 265
Validation binary_cross_entropy = 0.987903
Epoch 266
Validation binary_cross_entropy = 0.993230
Epoch 267
Validation binary_cross_entropy = 0.999391
Epoch 268
Validation binary_cross_entropy = 1.006427
Epoch 269
Loss = 4.8814e-04, PNorm = 105.3960, GNorm = 0.0456, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.009677
Epoch 270
Validation binary_cross_entropy = 1.010114
Epoch 271
Validation binary_cross_entropy = 0.993123
Epoch 272
Validation binary_cross_entropy = 0.978177
Epoch 273
Validation binary_cross_entropy = 0.963814
Epoch 274
Loss = 1.5569e-04, PNorm = 105.4119, GNorm = 0.0159, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.488521
Epoch 275
Validation binary_cross_entropy = 0.967972
Epoch 276
Validation binary_cross_entropy = 0.971811
Epoch 277
Validation binary_cross_entropy = 0.982068
Epoch 278
Validation binary_cross_entropy = 0.995477
Epoch 279
Loss = 1.8280e-04, PNorm = 105.4289, GNorm = 0.0103, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.010015
Epoch 280
Validation binary_cross_entropy = 1.029176
Epoch 281
Validation binary_cross_entropy = 1.046128
Epoch 282
Validation binary_cross_entropy = 1.060892
Epoch 283
Validation binary_cross_entropy = 1.081403
Epoch 284
Loss = 1.6676e-04, PNorm = 105.4423, GNorm = 0.0151, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.098340
Epoch 285
Validation binary_cross_entropy = 1.154442
Epoch 286
Validation binary_cross_entropy = 1.200051
Epoch 287
Validation binary_cross_entropy = 1.195227
Epoch 288
Validation binary_cross_entropy = 1.134880
Epoch 289
Loss = 2.9488e-04, PNorm = 105.4712, GNorm = 0.0184, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.090989
Epoch 290
Validation binary_cross_entropy = 1.059067
Epoch 291
Validation binary_cross_entropy = 1.043261
Epoch 292
Validation binary_cross_entropy = 1.039547
Epoch 293
Validation binary_cross_entropy = 1.059443
Epoch 294
Loss = 6.8536e-04, PNorm = 105.5064, GNorm = 0.0335, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.098609
Epoch 295
Validation binary_cross_entropy = 1.126669
Epoch 296
Validation binary_cross_entropy = 1.144281
Epoch 297
Validation binary_cross_entropy = 1.154491
Epoch 298
Validation binary_cross_entropy = 1.146104
Epoch 299
Loss = 3.6853e-04, PNorm = 105.5354, GNorm = 0.0399, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.126917
Model 0 best validation binary_cross_entropy = 0.289372 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.200234
Ensemble test binary_cross_entropy = 0.200234
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.092515
Epoch 1
Validation binary_cross_entropy = 0.300150
Epoch 2
Validation binary_cross_entropy = 1.825798
Epoch 3
Validation binary_cross_entropy = 0.550622
Epoch 4
Loss = 6.4268e-01, PNorm = 96.4275, GNorm = 7.0863, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.316422
Epoch 5
Validation binary_cross_entropy = 0.942584
Epoch 6
Validation binary_cross_entropy = 0.725171
Epoch 7
Validation binary_cross_entropy = 0.284973
Epoch 8
Validation binary_cross_entropy = 0.621314
Epoch 9
Loss = 5.3187e-01, PNorm = 96.7456, GNorm = 5.1313, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.781401
Epoch 10
Validation binary_cross_entropy = 0.330263
Epoch 11
Validation binary_cross_entropy = 0.396178
Epoch 12
Validation binary_cross_entropy = 0.476880
Epoch 13
Validation binary_cross_entropy = 0.337180
Epoch 14
Loss = 4.0650e-01, PNorm = 97.0669, GNorm = 3.0959, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.451481
Epoch 15
Validation binary_cross_entropy = 0.440683
Epoch 16
Validation binary_cross_entropy = 0.344005
Epoch 17
Validation binary_cross_entropy = 0.410889
Epoch 18
Validation binary_cross_entropy = 0.385593
Epoch 19
Loss = 2.5941e-01, PNorm = 97.3213, GNorm = 2.6554, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.450718
Epoch 20
Validation binary_cross_entropy = 0.421270
Epoch 21
Validation binary_cross_entropy = 0.452352
Epoch 22
Validation binary_cross_entropy = 0.386771
Epoch 23
Validation binary_cross_entropy = 0.651402
Epoch 24
Loss = 2.4974e-01, PNorm = 97.5613, GNorm = 6.0802, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.360213
Epoch 25
Validation binary_cross_entropy = 0.512991
Epoch 26
Validation binary_cross_entropy = 0.560426
Epoch 27
Validation binary_cross_entropy = 0.421605
Epoch 28
Validation binary_cross_entropy = 0.497180
Epoch 29
Loss = 2.6694e-01, PNorm = 97.8021, GNorm = 7.2241, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.558712
Epoch 30
Validation binary_cross_entropy = 0.419027
Epoch 31
Validation binary_cross_entropy = 0.456568
Epoch 32
Validation binary_cross_entropy = 0.481612
Epoch 33
Validation binary_cross_entropy = 0.438496
Epoch 34
Loss = 1.1853e-01, PNorm = 98.0394, GNorm = 1.4096, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.447641
Epoch 35
Validation binary_cross_entropy = 0.433903
Epoch 36
Validation binary_cross_entropy = 0.428824
Epoch 37
Validation binary_cross_entropy = 0.473649
Epoch 38
Validation binary_cross_entropy = 0.432348
Epoch 39
Loss = 1.7697e-01, PNorm = 98.2610, GNorm = 3.0465, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.437844
Epoch 40
Validation binary_cross_entropy = 0.406373
Epoch 41
Validation binary_cross_entropy = 0.409941
Epoch 42
Validation binary_cross_entropy = 0.407860
Epoch 43
Validation binary_cross_entropy = 0.438213
Epoch 44
Loss = 1.1428e-01, PNorm = 98.4847, GNorm = 1.8893, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.460514
Epoch 45
Validation binary_cross_entropy = 0.488298
Epoch 46
Validation binary_cross_entropy = 0.448847
Epoch 47
Validation binary_cross_entropy = 0.473069
Epoch 48
Validation binary_cross_entropy = 0.467188
Epoch 49
Loss = 6.8026e-02, PNorm = 98.6798, GNorm = 1.9083, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.556555
Epoch 50
Validation binary_cross_entropy = 0.511641
Epoch 51
Validation binary_cross_entropy = 0.560033
Epoch 52
Validation binary_cross_entropy = 0.651993
Epoch 53
Validation binary_cross_entropy = 0.752859
Epoch 54
Loss = 1.9764e-01, PNorm = 98.8812, GNorm = 2.9852, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.698257
Epoch 55
Validation binary_cross_entropy = 0.569285
Epoch 56
Validation binary_cross_entropy = 0.517470
Epoch 57
Validation binary_cross_entropy = 0.511800
Epoch 58
Validation binary_cross_entropy = 0.519274
Epoch 59
Loss = 6.9397e-02, PNorm = 99.1169, GNorm = 2.5894, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.566888
Epoch 60
Validation binary_cross_entropy = 0.581197
Epoch 61
Validation binary_cross_entropy = 0.582105
Epoch 62
Validation binary_cross_entropy = 0.528657
Epoch 63
Validation binary_cross_entropy = 0.580446
Epoch 64
Loss = 1.1742e-01, PNorm = 99.3292, GNorm = 2.9362, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.599386
Epoch 65
Validation binary_cross_entropy = 0.596384
Epoch 66
Validation binary_cross_entropy = 0.569979
Epoch 67
Validation binary_cross_entropy = 0.573856
Epoch 68
Validation binary_cross_entropy = 0.595441
Epoch 69
Loss = 9.5048e-02, PNorm = 99.5511, GNorm = 2.1421, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.597655
Epoch 70
Validation binary_cross_entropy = 0.582750
Epoch 71
Validation binary_cross_entropy = 0.546162
Epoch 72
Validation binary_cross_entropy = 0.574203
Epoch 73
Validation binary_cross_entropy = 0.578716
Epoch 74
Loss = 1.0629e-01, PNorm = 99.7692, GNorm = 2.8929, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.616918
Epoch 75
Validation binary_cross_entropy = 0.580380
Epoch 76
Validation binary_cross_entropy = 0.654643
Epoch 77
Validation binary_cross_entropy = 0.629085
Epoch 78
Validation binary_cross_entropy = 0.591444
Epoch 79
Loss = 6.5291e-02, PNorm = 99.9896, GNorm = 2.3824, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.601466
Epoch 80
Validation binary_cross_entropy = 0.657872
Epoch 81
Validation binary_cross_entropy = 0.611140
Epoch 82
Validation binary_cross_entropy = 0.605545
Epoch 83
Validation binary_cross_entropy = 0.635771
Epoch 84
Loss = 2.1599e-02, PNorm = 100.2091, GNorm = 1.6111, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.654164
Epoch 85
Validation binary_cross_entropy = 0.692464
Epoch 86
Validation binary_cross_entropy = 0.670595
Epoch 87
Validation binary_cross_entropy = 0.686741
Epoch 88
Validation binary_cross_entropy = 0.683676
Epoch 89
Loss = 1.0276e-01, PNorm = 100.4124, GNorm = 2.6258, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.673417
Epoch 90
Validation binary_cross_entropy = 0.667495
Epoch 91
Validation binary_cross_entropy = 0.818363
Epoch 92
Validation binary_cross_entropy = 0.678953
Epoch 93
Validation binary_cross_entropy = 0.823432
Epoch 94
Loss = 1.5947e-01, PNorm = 100.6457, GNorm = 3.5049, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.965661
Epoch 95
Validation binary_cross_entropy = 0.706931
Epoch 96
Validation binary_cross_entropy = 1.060403
Epoch 97
Validation binary_cross_entropy = 0.565209
Epoch 98
Validation binary_cross_entropy = 0.626077
Epoch 99
Loss = 1.7641e-01, PNorm = 100.9658, GNorm = 1.8956, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.623637
Epoch 100
Validation binary_cross_entropy = 0.775939
Epoch 101
Validation binary_cross_entropy = 0.587203
Epoch 102
Validation binary_cross_entropy = 0.620450
Epoch 103
Validation binary_cross_entropy = 0.577240
Epoch 104
Loss = 7.0522e-02, PNorm = 101.3649, GNorm = 1.2901, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.548569
Epoch 105
Validation binary_cross_entropy = 0.548516
Epoch 106
Validation binary_cross_entropy = 0.503255
Epoch 107
Validation binary_cross_entropy = 0.491503
Epoch 108
Validation binary_cross_entropy = 0.502536
Epoch 109
Loss = 3.8317e-02, PNorm = 101.6677, GNorm = 1.2653, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.504790
Epoch 110
Validation binary_cross_entropy = 0.521669
Epoch 111
Validation binary_cross_entropy = 0.538992
Epoch 112
Validation binary_cross_entropy = 0.619121
Epoch 113
Validation binary_cross_entropy = 0.618401
Epoch 114
Loss = 4.0158e-02, PNorm = 101.9207, GNorm = 2.1433, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.560329
Epoch 115
Validation binary_cross_entropy = 0.547018
Epoch 116
Validation binary_cross_entropy = 0.560140
Epoch 117
Validation binary_cross_entropy = 0.607410
Epoch 118
Validation binary_cross_entropy = 0.648453
Epoch 119
Loss = 5.1253e-02, PNorm = 102.1358, GNorm = 2.1018, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.641253
Epoch 120
Validation binary_cross_entropy = 0.646352
Epoch 121
Validation binary_cross_entropy = 0.639865
Epoch 122
Validation binary_cross_entropy = 0.633168
Epoch 123
Validation binary_cross_entropy = 0.640582
Epoch 124
Loss = 1.0591e-02, PNorm = 102.3079, GNorm = 0.8364, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.650968
Epoch 125
Validation binary_cross_entropy = 0.701376
Epoch 126
Validation binary_cross_entropy = 0.697240
Epoch 127
Validation binary_cross_entropy = 0.666219
Epoch 128
Validation binary_cross_entropy = 0.674346
Epoch 129
Loss = 9.5941e-03, PNorm = 102.4429, GNorm = 0.6921, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.694523
Epoch 130
Validation binary_cross_entropy = 0.721989
Epoch 131
Validation binary_cross_entropy = 0.718908
Epoch 132
Validation binary_cross_entropy = 0.720811
Epoch 133
Validation binary_cross_entropy = 0.739431
Epoch 134
Loss = 4.1362e-03, PNorm = 102.5518, GNorm = 0.3823, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.755869
Epoch 135
Validation binary_cross_entropy = 0.767674
Epoch 136
Validation binary_cross_entropy = 0.768945
Epoch 137
Validation binary_cross_entropy = 0.801552
Epoch 138
Validation binary_cross_entropy = 0.842514
Epoch 139
Loss = 5.9185e-03, PNorm = 102.6436, GNorm = 0.2252, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.844420
Epoch 140
Validation binary_cross_entropy = 0.819200
Epoch 141
Validation binary_cross_entropy = 0.802684
Epoch 142
Validation binary_cross_entropy = 0.799409
Epoch 143
Validation binary_cross_entropy = 0.808343
Epoch 144
Loss = 8.1652e-03, PNorm = 102.7066, GNorm = 1.0176, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.852620
Epoch 145
Validation binary_cross_entropy = 0.941145
Epoch 146
Validation binary_cross_entropy = 0.953002
Epoch 147
Validation binary_cross_entropy = 0.878905
Epoch 148
Validation binary_cross_entropy = 0.818944
Epoch 149
Loss = 2.9697e-02, PNorm = 102.7997, GNorm = 2.6915, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.838577
Epoch 150
Validation binary_cross_entropy = 0.889185
Epoch 151
Validation binary_cross_entropy = 0.933779
Epoch 152
Validation binary_cross_entropy = 0.951996
Epoch 153
Validation binary_cross_entropy = 1.038965
Epoch 154
Loss = 1.5530e-02, PNorm = 102.9159, GNorm = 0.4381, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.940481
Epoch 155
Validation binary_cross_entropy = 0.860293
Epoch 156
Validation binary_cross_entropy = 0.833272
Epoch 157
Validation binary_cross_entropy = 0.830500
Epoch 158
Validation binary_cross_entropy = 0.848681
Epoch 159
Loss = 1.5115e-02, PNorm = 103.0502, GNorm = 1.7074, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.860440
Epoch 160
Validation binary_cross_entropy = 0.856197
Epoch 161
Validation binary_cross_entropy = 0.878649
Epoch 162
Validation binary_cross_entropy = 0.889716
Epoch 163
Validation binary_cross_entropy = 0.865541
Epoch 164
Loss = 7.2604e-03, PNorm = 103.1776, GNorm = 0.8518, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.846382
Epoch 165
Validation binary_cross_entropy = 0.829195
Epoch 166
Validation binary_cross_entropy = 0.880329
Epoch 167
Validation binary_cross_entropy = 0.916699
Epoch 168
Validation binary_cross_entropy = 0.855563
Epoch 169
Loss = 7.0434e-03, PNorm = 103.3132, GNorm = 0.5090, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.849676
Epoch 170
Validation binary_cross_entropy = 0.856060
Epoch 171
Validation binary_cross_entropy = 0.884214
Epoch 172
Validation binary_cross_entropy = 0.923866
Epoch 173
Validation binary_cross_entropy = 0.904094
Epoch 174
Loss = 1.7715e-03, PNorm = 103.4217, GNorm = 0.2699, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.878904
Epoch 175
Validation binary_cross_entropy = 0.881651
Epoch 176
Validation binary_cross_entropy = 0.904575
Epoch 177
Validation binary_cross_entropy = 0.914165
Epoch 178
Validation binary_cross_entropy = 0.903873
Epoch 179
Loss = 6.8762e-03, PNorm = 103.5213, GNorm = 1.1927, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.909562
Epoch 180
Validation binary_cross_entropy = 0.952546
Epoch 181
Validation binary_cross_entropy = 0.993607
Epoch 182
Validation binary_cross_entropy = 0.961824
Epoch 183
Validation binary_cross_entropy = 1.017303
Epoch 184
Loss = 2.1270e-03, PNorm = 103.6122, GNorm = 0.0642, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.062925
Epoch 185
Validation binary_cross_entropy = 1.054103
Epoch 186
Validation binary_cross_entropy = 1.055194
Epoch 187
Validation binary_cross_entropy = 1.075898
Epoch 188
Validation binary_cross_entropy = 1.066721
Epoch 189
Loss = 8.4899e-04, PNorm = 103.7042, GNorm = 0.0431, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.029052
Epoch 190
Validation binary_cross_entropy = 1.004727
Epoch 191
Validation binary_cross_entropy = 0.985678
Epoch 192
Validation binary_cross_entropy = 0.981311
Epoch 193
Validation binary_cross_entropy = 0.983965
Epoch 194
Loss = 3.0714e-03, PNorm = 103.7552, GNorm = 0.5753, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.995247
Epoch 195
Validation binary_cross_entropy = 1.040821
Epoch 196
Validation binary_cross_entropy = 0.989329
Epoch 197
Validation binary_cross_entropy = 0.964504
Epoch 198
Validation binary_cross_entropy = 0.987633
Epoch 199
Loss = 4.7696e-03, PNorm = 103.8313, GNorm = 0.1267, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.017311
Epoch 200
Validation binary_cross_entropy = 0.973252
Epoch 201
Validation binary_cross_entropy = 0.980155
Epoch 202
Validation binary_cross_entropy = 1.149418
Epoch 203
Validation binary_cross_entropy = 1.007111
Epoch 204
Loss = 2.4476e-02, PNorm = 103.9886, GNorm = 0.5394, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.205363
Epoch 205
Validation binary_cross_entropy = 1.429635
Epoch 206
Validation binary_cross_entropy = 1.476879
Epoch 207
Validation binary_cross_entropy = 1.478908
Epoch 208
Validation binary_cross_entropy = 0.879004
Epoch 209
Loss = 1.6944e-02, PNorm = 104.2296, GNorm = 0.5614, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.049088
Epoch 210
Validation binary_cross_entropy = 0.977038
Epoch 211
Validation binary_cross_entropy = 0.872724
Epoch 212
Validation binary_cross_entropy = 0.868774
Epoch 213
Validation binary_cross_entropy = 0.896874
Epoch 214
Loss = 5.5968e-03, PNorm = 104.4482, GNorm = 0.5245, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.961403
Epoch 215
Validation binary_cross_entropy = 1.452875
Epoch 216
Validation binary_cross_entropy = 1.405692
Epoch 217
Validation binary_cross_entropy = 1.426480
Epoch 218
Validation binary_cross_entropy = 0.936110
Epoch 219
Loss = 7.6208e-03, PNorm = 104.6406, GNorm = 0.9827, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.026704
Epoch 220
Validation binary_cross_entropy = 0.945499
Epoch 221
Validation binary_cross_entropy = 0.970615
Epoch 222
Validation binary_cross_entropy = 0.991188
Epoch 223
Validation binary_cross_entropy = 0.930531
Epoch 224
Loss = 4.6880e-02, PNorm = 104.7842, GNorm = 0.4053, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.968517
Epoch 225
Validation binary_cross_entropy = 1.014531
Epoch 226
Validation binary_cross_entropy = 0.966516
Epoch 227
Validation binary_cross_entropy = 0.932452
Epoch 228
Validation binary_cross_entropy = 0.923311
Epoch 229
Loss = 5.7364e-04, PNorm = 104.9021, GNorm = 0.0542, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.938623
Epoch 230
Validation binary_cross_entropy = 0.974210
Epoch 231
Validation binary_cross_entropy = 0.996729
Epoch 232
Validation binary_cross_entropy = 0.975581
Epoch 233
Validation binary_cross_entropy = 0.942484
Epoch 234
Loss = 1.0362e-02, PNorm = 105.0121, GNorm = 0.6060, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.940816
Epoch 235
Validation binary_cross_entropy = 0.964435
Epoch 236
Validation binary_cross_entropy = 1.015669
Epoch 237
Validation binary_cross_entropy = 1.046358
Epoch 238
Validation binary_cross_entropy = 0.954671
Epoch 239
Loss = 8.9748e-03, PNorm = 105.1656, GNorm = 1.6303, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.928474
Epoch 240
Validation binary_cross_entropy = 0.934358
Epoch 241
Validation binary_cross_entropy = 1.010788
Epoch 242
Validation binary_cross_entropy = 0.973952
Epoch 243
Validation binary_cross_entropy = 0.959908
Epoch 244
Loss = 1.4374e-03, PNorm = 105.2949, GNorm = 0.2444, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.492913
Epoch 245
Validation binary_cross_entropy = 1.492619
Epoch 246
Validation binary_cross_entropy = 1.495584
Epoch 247
Validation binary_cross_entropy = 1.515749
Epoch 248
Validation binary_cross_entropy = 1.075585
Epoch 249
Loss = 3.4055e-02, PNorm = 105.4040, GNorm = 3.3037, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.091910
Epoch 250
Validation binary_cross_entropy = 1.061603
Epoch 251
Validation binary_cross_entropy = 1.044081
Epoch 252
Validation binary_cross_entropy = 1.038038
Epoch 253
Validation binary_cross_entropy = 1.034161
Epoch 254
Loss = 3.8588e-02, PNorm = 105.4913, GNorm = 3.5810, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.043096
Epoch 255
Validation binary_cross_entropy = 1.103297
Epoch 256
Validation binary_cross_entropy = 1.128352
Epoch 257
Validation binary_cross_entropy = 1.103332
Epoch 258
Validation binary_cross_entropy = 1.077160
Epoch 259
Loss = 4.4876e-03, PNorm = 105.5655, GNorm = 0.1413, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.025479
Epoch 260
Validation binary_cross_entropy = 1.516882
Epoch 261
Validation binary_cross_entropy = 1.490514
Epoch 262
Validation binary_cross_entropy = 1.482592
Epoch 263
Validation binary_cross_entropy = 1.478378
Epoch 264
Loss = 4.6085e-04, PNorm = 105.6165, GNorm = 0.0404, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.476426
Epoch 265
Validation binary_cross_entropy = 1.476732
Epoch 266
Validation binary_cross_entropy = 1.478263
Epoch 267
Validation binary_cross_entropy = 1.482785
Epoch 268
Validation binary_cross_entropy = 1.485317
Epoch 269
Loss = 1.4016e-04, PNorm = 105.6504, GNorm = 0.0056, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.489879
Epoch 270
Validation binary_cross_entropy = 1.492037
Epoch 271
Validation binary_cross_entropy = 1.495670
Epoch 272
Validation binary_cross_entropy = 1.502435
Epoch 273
Validation binary_cross_entropy = 1.508225
Epoch 274
Loss = 9.3607e-04, PNorm = 105.6754, GNorm = 0.0180, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.518838
Epoch 275
Validation binary_cross_entropy = 1.528806
Epoch 276
Validation binary_cross_entropy = 1.533763
Epoch 277
Validation binary_cross_entropy = 1.537486
Epoch 278
Validation binary_cross_entropy = 1.551887
Epoch 279
Loss = 4.3903e-04, PNorm = 105.6922, GNorm = 0.0052, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.559684
Epoch 280
Validation binary_cross_entropy = 1.561271
Epoch 281
Validation binary_cross_entropy = 1.563631
Epoch 282
Validation binary_cross_entropy = 1.566564
Epoch 283
Validation binary_cross_entropy = 1.568203
Epoch 284
Loss = 1.7314e-04, PNorm = 105.7029, GNorm = 0.0224, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.568499
Epoch 285
Validation binary_cross_entropy = 1.557405
Epoch 286
Validation binary_cross_entropy = 1.543390
Epoch 287
Validation binary_cross_entropy = 1.539350
Epoch 288
Validation binary_cross_entropy = 1.536982
Epoch 289
Loss = 1.5250e-02, PNorm = 105.7123, GNorm = 0.0024, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.557346
Epoch 290
Validation binary_cross_entropy = 1.608140
Epoch 291
Validation binary_cross_entropy = 1.657133
Epoch 292
Validation binary_cross_entropy = 1.681045
Epoch 293
Validation binary_cross_entropy = 1.675115
Epoch 294
Loss = 2.3430e-03, PNorm = 105.7372, GNorm = 0.0556, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.648010
Epoch 295
Validation binary_cross_entropy = 1.621675
Epoch 296
Validation binary_cross_entropy = 1.600758
Epoch 297
Validation binary_cross_entropy = 1.585463
Epoch 298
Validation binary_cross_entropy = 1.563917
Epoch 299
Loss = 1.9181e-04, PNorm = 105.7621, GNorm = 0.0180, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.548288
Model 0 best validation binary_cross_entropy = 0.284973 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.221389
Ensemble test binary_cross_entropy = 0.221389
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.051729
Epoch 1
Validation binary_cross_entropy = 0.278861
Epoch 2
Validation binary_cross_entropy = 1.364050
Epoch 3
Validation binary_cross_entropy = 0.878749
Epoch 4
Loss = 6.1889e-01, PNorm = 96.4334, GNorm = 4.3471, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.294405
Epoch 5
Validation binary_cross_entropy = 0.708080
Epoch 6
Validation binary_cross_entropy = 1.045140
Epoch 7
Validation binary_cross_entropy = 0.308041
Epoch 8
Validation binary_cross_entropy = 0.452769
Epoch 9
Loss = 5.1722e-01, PNorm = 96.7623, GNorm = 3.1500, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.931127
Epoch 10
Validation binary_cross_entropy = 0.331655
Epoch 11
Validation binary_cross_entropy = 0.456895
Epoch 12
Validation binary_cross_entropy = 0.427624
Epoch 13
Validation binary_cross_entropy = 0.369879
Epoch 14
Loss = 3.7952e-01, PNorm = 97.0913, GNorm = 2.9532, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.614526
Epoch 15
Validation binary_cross_entropy = 0.378826
Epoch 16
Validation binary_cross_entropy = 0.692836
Epoch 17
Validation binary_cross_entropy = 0.997206
Epoch 18
Validation binary_cross_entropy = 0.339158
Epoch 19
Loss = 3.0670e-01, PNorm = 97.3407, GNorm = 4.5657, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.467606
Epoch 20
Validation binary_cross_entropy = 0.674443
Epoch 21
Validation binary_cross_entropy = 0.320233
Epoch 22
Validation binary_cross_entropy = 0.385174
Epoch 23
Validation binary_cross_entropy = 0.671801
Epoch 24
Loss = 3.0975e-01, PNorm = 97.5667, GNorm = 2.5866, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.363656
Epoch 25
Validation binary_cross_entropy = 0.427961
Epoch 26
Validation binary_cross_entropy = 0.478288
Epoch 27
Validation binary_cross_entropy = 0.506209
Epoch 28
Validation binary_cross_entropy = 0.532797
Epoch 29
Loss = 3.0051e-01, PNorm = 97.7794, GNorm = 4.0489, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.349086
Epoch 30
Validation binary_cross_entropy = 0.443080
Epoch 31
Validation binary_cross_entropy = 0.370880
Epoch 32
Validation binary_cross_entropy = 0.361415
Epoch 33
Validation binary_cross_entropy = 0.384902
Epoch 34
Loss = 1.8344e-01, PNorm = 98.0049, GNorm = 1.9958, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.394252
Epoch 35
Validation binary_cross_entropy = 0.438783
Epoch 36
Validation binary_cross_entropy = 0.501877
Epoch 37
Validation binary_cross_entropy = 0.566940
Epoch 38
Validation binary_cross_entropy = 0.638886
Epoch 39
Loss = 2.3425e-01, PNorm = 98.2199, GNorm = 6.3173, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.506852
Epoch 40
Validation binary_cross_entropy = 0.454313
Epoch 41
Validation binary_cross_entropy = 0.440483
Epoch 42
Validation binary_cross_entropy = 0.401405
Epoch 43
Validation binary_cross_entropy = 0.422206
Epoch 44
Loss = 3.6942e-01, PNorm = 98.4411, GNorm = 6.3957, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.537122
Epoch 45
Validation binary_cross_entropy = 0.434114
Epoch 46
Validation binary_cross_entropy = 0.466421
Epoch 47
Validation binary_cross_entropy = 0.444060
Epoch 48
Validation binary_cross_entropy = 0.446888
Epoch 49
Loss = 1.5811e-01, PNorm = 98.6756, GNorm = 3.6995, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.439048
Epoch 50
Validation binary_cross_entropy = 0.434275
Epoch 51
Validation binary_cross_entropy = 0.624371
Epoch 52
Validation binary_cross_entropy = 0.477752
Epoch 53
Validation binary_cross_entropy = 0.486745
Epoch 54
Loss = 1.1251e-01, PNorm = 98.9097, GNorm = 1.4707, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.479607
Epoch 55
Validation binary_cross_entropy = 0.466428
Epoch 56
Validation binary_cross_entropy = 0.455353
Epoch 57
Validation binary_cross_entropy = 0.453636
Epoch 58
Validation binary_cross_entropy = 0.529240
Epoch 59
Loss = 1.5537e-01, PNorm = 99.1403, GNorm = 3.5724, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.487319
Epoch 60
Validation binary_cross_entropy = 0.479652
Epoch 61
Validation binary_cross_entropy = 0.500031
Epoch 62
Validation binary_cross_entropy = 0.508498
Epoch 63
Validation binary_cross_entropy = 0.466812
Epoch 64
Loss = 1.0542e-01, PNorm = 99.3545, GNorm = 3.5879, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.451130
Epoch 65
Validation binary_cross_entropy = 0.457172
Epoch 66
Validation binary_cross_entropy = 0.479388
Epoch 67
Validation binary_cross_entropy = 0.485861
Epoch 68
Validation binary_cross_entropy = 0.548928
Epoch 69
Loss = 4.3871e-02, PNorm = 99.5583, GNorm = 2.5725, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.572605
Epoch 70
Validation binary_cross_entropy = 0.581634
Epoch 71
Validation binary_cross_entropy = 0.550593
Epoch 72
Validation binary_cross_entropy = 0.545969
Epoch 73
Validation binary_cross_entropy = 0.559703
Epoch 74
Loss = 2.1212e-01, PNorm = 99.7947, GNorm = 8.0746, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.537359
Epoch 75
Validation binary_cross_entropy = 0.489308
Epoch 76
Validation binary_cross_entropy = 0.501783
Epoch 77
Validation binary_cross_entropy = 0.518419
Epoch 78
Validation binary_cross_entropy = 0.510660
Epoch 79
Loss = 2.0004e-01, PNorm = 100.0929, GNorm = 5.0332, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.503140
Epoch 80
Validation binary_cross_entropy = 0.535668
Epoch 81
Validation binary_cross_entropy = 0.515385
Epoch 82
Validation binary_cross_entropy = 0.527509
Epoch 83
Validation binary_cross_entropy = 0.524339
Epoch 84
Loss = 1.0375e-01, PNorm = 100.3764, GNorm = 2.1719, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.497956
Epoch 85
Validation binary_cross_entropy = 0.493049
Epoch 86
Validation binary_cross_entropy = 0.507986
Epoch 87
Validation binary_cross_entropy = 0.512916
Epoch 88
Validation binary_cross_entropy = 0.522900
Epoch 89
Loss = 5.9436e-02, PNorm = 100.6458, GNorm = 1.8239, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.527873
Epoch 90
Validation binary_cross_entropy = 0.527458
Epoch 91
Validation binary_cross_entropy = 0.528704
Epoch 92
Validation binary_cross_entropy = 0.548732
Epoch 93
Validation binary_cross_entropy = 0.591793
Epoch 94
Loss = 4.2244e-02, PNorm = 100.8641, GNorm = 2.7747, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.564955
Epoch 95
Validation binary_cross_entropy = 0.568875
Epoch 96
Validation binary_cross_entropy = 0.603847
Epoch 97
Validation binary_cross_entropy = 0.636781
Epoch 98
Validation binary_cross_entropy = 0.621986
Epoch 99
Loss = 7.8269e-02, PNorm = 101.0282, GNorm = 3.8883, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.647406
Epoch 100
Validation binary_cross_entropy = 0.683927
Epoch 101
Validation binary_cross_entropy = 0.671350
Epoch 102
Validation binary_cross_entropy = 0.672279
Epoch 103
Validation binary_cross_entropy = 0.670027
Epoch 104
Loss = 4.2309e-02, PNorm = 101.2152, GNorm = 0.7333, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.761196
Epoch 105
Validation binary_cross_entropy = 0.702865
Epoch 106
Validation binary_cross_entropy = 0.697416
Epoch 107
Validation binary_cross_entropy = 0.697271
Epoch 108
Validation binary_cross_entropy = 0.800293
Epoch 109
Loss = 8.8031e-02, PNorm = 101.3744, GNorm = 1.6635, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.729038
Epoch 110
Validation binary_cross_entropy = 0.664905
Epoch 111
Validation binary_cross_entropy = 0.696226
Epoch 112
Validation binary_cross_entropy = 0.733560
Epoch 113
Validation binary_cross_entropy = 0.704933
Epoch 114
Loss = 3.0177e-02, PNorm = 101.5234, GNorm = 1.3371, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.654093
Epoch 115
Validation binary_cross_entropy = 0.641702
Epoch 116
Validation binary_cross_entropy = 0.697718
Epoch 117
Validation binary_cross_entropy = 0.710581
Epoch 118
Validation binary_cross_entropy = 0.692338
Epoch 119
Loss = 4.2559e-02, PNorm = 101.6719, GNorm = 4.6275, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.704733
Epoch 120
Validation binary_cross_entropy = 0.742071
Epoch 121
Validation binary_cross_entropy = 0.798382
Epoch 122
Validation binary_cross_entropy = 0.704611
Epoch 123
Validation binary_cross_entropy = 0.677500
Epoch 124
Loss = 1.4001e-02, PNorm = 101.8419, GNorm = 1.0481, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.673182
Epoch 125
Validation binary_cross_entropy = 0.698715
Epoch 126
Validation binary_cross_entropy = 0.700956
Epoch 127
Validation binary_cross_entropy = 0.722308
Epoch 128
Validation binary_cross_entropy = 0.699429
Epoch 129
Loss = 8.0579e-03, PNorm = 101.9790, GNorm = 0.9572, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.682454
Epoch 130
Validation binary_cross_entropy = 0.725913
Epoch 131
Validation binary_cross_entropy = 0.742719
Epoch 132
Validation binary_cross_entropy = 0.710020
Epoch 133
Validation binary_cross_entropy = 0.693152
Epoch 134
Loss = 1.7402e-02, PNorm = 102.1010, GNorm = 2.1279, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.708397
Epoch 135
Validation binary_cross_entropy = 0.727580
Epoch 136
Validation binary_cross_entropy = 0.817820
Epoch 137
Validation binary_cross_entropy = 0.793077
Epoch 138
Validation binary_cross_entropy = 0.784975
Epoch 139
Loss = 5.0151e-02, PNorm = 102.2439, GNorm = 2.9503, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.842971
Epoch 140
Validation binary_cross_entropy = 0.861423
Epoch 141
Validation binary_cross_entropy = 0.907273
Epoch 142
Validation binary_cross_entropy = 0.838739
Epoch 143
Validation binary_cross_entropy = 0.808137
Epoch 144
Loss = 7.3119e-02, PNorm = 102.4135, GNorm = 0.4472, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.849035
Epoch 145
Validation binary_cross_entropy = 0.875812
Epoch 146
Validation binary_cross_entropy = 0.858192
Epoch 147
Validation binary_cross_entropy = 0.824517
Epoch 148
Validation binary_cross_entropy = 0.796132
Epoch 149
Loss = 1.2068e-02, PNorm = 102.5958, GNorm = 1.4533, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.776505
Epoch 150
Validation binary_cross_entropy = 0.763455
Epoch 151
Validation binary_cross_entropy = 0.753134
Epoch 152
Validation binary_cross_entropy = 0.748638
Epoch 153
Validation binary_cross_entropy = 0.760841
Epoch 154
Loss = 3.6168e-02, PNorm = 102.7199, GNorm = 0.6929, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.810339
Epoch 155
Validation binary_cross_entropy = 0.801208
Epoch 156
Validation binary_cross_entropy = 0.783918
Epoch 157
Validation binary_cross_entropy = 0.755219
Epoch 158
Validation binary_cross_entropy = 0.801185
Epoch 159
Loss = 2.8735e-02, PNorm = 102.8375, GNorm = 0.2951, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.885209
Epoch 160
Validation binary_cross_entropy = 0.889402
Epoch 161
Validation binary_cross_entropy = 0.807205
Epoch 162
Validation binary_cross_entropy = 0.779293
Epoch 163
Validation binary_cross_entropy = 0.796650
Epoch 164
Loss = 4.2364e-03, PNorm = 102.9324, GNorm = 0.2334, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.821174
Epoch 165
Validation binary_cross_entropy = 0.867740
Epoch 166
Validation binary_cross_entropy = 0.890332
Epoch 167
Validation binary_cross_entropy = 0.876086
Epoch 168
Validation binary_cross_entropy = 0.853555
Epoch 169
Loss = 1.3484e-02, PNorm = 103.0362, GNorm = 0.1461, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.873000
Epoch 170
Validation binary_cross_entropy = 0.882156
Epoch 171
Validation binary_cross_entropy = 0.896896
Epoch 172
Validation binary_cross_entropy = 0.912307
Epoch 173
Validation binary_cross_entropy = 0.928398
Epoch 174
Loss = 7.8143e-03, PNorm = 103.1180, GNorm = 0.0396, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.958802
Epoch 175
Validation binary_cross_entropy = 0.971681
Epoch 176
Validation binary_cross_entropy = 1.000228
Epoch 177
Validation binary_cross_entropy = 1.003711
Epoch 178
Validation binary_cross_entropy = 0.991490
Epoch 179
Loss = 3.4379e-03, PNorm = 103.1876, GNorm = 0.1137, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.980650
Epoch 180
Validation binary_cross_entropy = 0.975409
Epoch 181
Validation binary_cross_entropy = 0.971952
Epoch 182
Validation binary_cross_entropy = 0.970576
Epoch 183
Validation binary_cross_entropy = 0.973462
Epoch 184
Loss = 8.1990e-03, PNorm = 103.2423, GNorm = 0.0132, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.024451
Epoch 185
Validation binary_cross_entropy = 1.066585
Epoch 186
Validation binary_cross_entropy = 1.061086
Epoch 187
Validation binary_cross_entropy = 1.062394
Epoch 188
Validation binary_cross_entropy = 1.036132
Epoch 189
Loss = 2.3928e-03, PNorm = 103.2946, GNorm = 0.0312, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.990010
Epoch 190
Validation binary_cross_entropy = 0.960022
Epoch 191
Validation binary_cross_entropy = 1.042468
Epoch 192
Validation binary_cross_entropy = 1.058557
Epoch 193
Validation binary_cross_entropy = 1.019510
Epoch 194
Loss = 2.5005e-03, PNorm = 103.3534, GNorm = 0.0848, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.967810
Epoch 195
Validation binary_cross_entropy = 0.941580
Epoch 196
Validation binary_cross_entropy = 0.937075
Epoch 197
Validation binary_cross_entropy = 0.953395
Epoch 198
Validation binary_cross_entropy = 1.046006
Epoch 199
Loss = 4.4891e-02, PNorm = 103.4215, GNorm = 0.7638, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.993856
Epoch 200
Validation binary_cross_entropy = 0.954166
Epoch 201
Validation binary_cross_entropy = 0.992842
Epoch 202
Validation binary_cross_entropy = 0.949457
Epoch 203
Validation binary_cross_entropy = 0.996535
Epoch 204
Loss = 8.9694e-02, PNorm = 103.5321, GNorm = 4.0808, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.025884
Epoch 205
Validation binary_cross_entropy = 0.981456
Epoch 206
Validation binary_cross_entropy = 0.914169
Epoch 207
Validation binary_cross_entropy = 0.878929
Epoch 208
Validation binary_cross_entropy = 0.896072
Epoch 209
Loss = 1.1738e-02, PNorm = 103.7145, GNorm = 1.8048, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.885294
Epoch 210
Validation binary_cross_entropy = 1.005382
Epoch 211
Validation binary_cross_entropy = 0.946683
Epoch 212
Validation binary_cross_entropy = 0.919931
Epoch 213
Validation binary_cross_entropy = 0.906291
Epoch 214
Loss = 7.9465e-02, PNorm = 103.8885, GNorm = 3.5040, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.042207
Epoch 215
Validation binary_cross_entropy = 0.995849
Epoch 216
Validation binary_cross_entropy = 1.431261
Epoch 217
Validation binary_cross_entropy = 1.511842
Epoch 218
Validation binary_cross_entropy = 1.509462
Epoch 219
Loss = 6.6261e-02, PNorm = 104.1792, GNorm = 4.0234, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.440685
Epoch 220
Validation binary_cross_entropy = 0.865044
Epoch 221
Validation binary_cross_entropy = 0.978304
Epoch 222
Validation binary_cross_entropy = 1.102957
Epoch 223
Validation binary_cross_entropy = 1.037885
Epoch 224
Loss = 2.8337e-02, PNorm = 104.5035, GNorm = 1.7620, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.049287
Epoch 225
Validation binary_cross_entropy = 1.074275
Epoch 226
Validation binary_cross_entropy = 0.990400
Epoch 227
Validation binary_cross_entropy = 0.939836
Epoch 228
Validation binary_cross_entropy = 0.939653
Epoch 229
Loss = 2.7256e-02, PNorm = 104.7748, GNorm = 2.2271, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.963021
Epoch 230
Validation binary_cross_entropy = 0.976519
Epoch 231
Validation binary_cross_entropy = 0.986558
Epoch 232
Validation binary_cross_entropy = 0.955671
Epoch 233
Validation binary_cross_entropy = 0.949062
Epoch 234
Loss = 7.6285e-03, PNorm = 104.9448, GNorm = 0.5458, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.948065
Epoch 235
Validation binary_cross_entropy = 0.956713
Epoch 236
Validation binary_cross_entropy = 1.019569
Epoch 237
Validation binary_cross_entropy = 1.151078
Epoch 238
Validation binary_cross_entropy = 1.037280
Epoch 239
Loss = 1.9829e-02, PNorm = 105.0917, GNorm = 0.1816, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.961648
Epoch 240
Validation binary_cross_entropy = 0.953292
Epoch 241
Validation binary_cross_entropy = 0.957059
Epoch 242
Validation binary_cross_entropy = 0.959539
Epoch 243
Validation binary_cross_entropy = 0.976342
Epoch 244
Loss = 1.6033e-02, PNorm = 105.2648, GNorm = 0.7287, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.998792
Epoch 245
Validation binary_cross_entropy = 1.002695
Epoch 246
Validation binary_cross_entropy = 1.005563
Epoch 247
Validation binary_cross_entropy = 1.009687
Epoch 248
Validation binary_cross_entropy = 1.017431
Epoch 249
Loss = 3.4465e-03, PNorm = 105.3677, GNorm = 0.1520, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.052323
Epoch 250
Validation binary_cross_entropy = 1.074279
Epoch 251
Validation binary_cross_entropy = 1.043398
Epoch 252
Validation binary_cross_entropy = 1.025577
Epoch 253
Validation binary_cross_entropy = 1.018543
Epoch 254
Loss = 4.4242e-03, PNorm = 105.4412, GNorm = 0.4929, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.013690
Epoch 255
Validation binary_cross_entropy = 1.014605
Epoch 256
Validation binary_cross_entropy = 1.026042
Epoch 257
Validation binary_cross_entropy = 1.036360
Epoch 258
Validation binary_cross_entropy = 1.039977
Epoch 259
Loss = 7.3194e-04, PNorm = 105.4889, GNorm = 0.1036, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.041979
Epoch 260
Validation binary_cross_entropy = 1.049465
Epoch 261
Validation binary_cross_entropy = 1.118152
Epoch 262
Validation binary_cross_entropy = 1.129177
Epoch 263
Validation binary_cross_entropy = 1.094965
Epoch 264
Loss = 2.9807e-03, PNorm = 105.5377, GNorm = 0.2815, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.043435
Epoch 265
Validation binary_cross_entropy = 0.993441
Epoch 266
Validation binary_cross_entropy = 0.965577
Epoch 267
Validation binary_cross_entropy = 0.957880
Epoch 268
Validation binary_cross_entropy = 0.959860
Epoch 269
Loss = 9.7067e-04, PNorm = 105.5666, GNorm = 0.0130, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.985596
Epoch 270
Validation binary_cross_entropy = 1.011701
Epoch 271
Validation binary_cross_entropy = 1.038368
Epoch 272
Validation binary_cross_entropy = 1.052700
Epoch 273
Validation binary_cross_entropy = 0.990405
Epoch 274
Loss = 1.0150e-03, PNorm = 105.5854, GNorm = 0.1331, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.962032
Epoch 275
Validation binary_cross_entropy = 0.957289
Epoch 276
Validation binary_cross_entropy = 0.960703
Epoch 277
Validation binary_cross_entropy = 0.984876
Epoch 278
Validation binary_cross_entropy = 1.036393
Epoch 279
Loss = 5.0370e-03, PNorm = 105.6120, GNorm = 0.4994, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.052059
Epoch 280
Validation binary_cross_entropy = 1.055559
Epoch 281
Validation binary_cross_entropy = 1.054050
Epoch 282
Validation binary_cross_entropy = 1.048098
Epoch 283
Validation binary_cross_entropy = 1.048678
Epoch 284
Loss = 4.0717e-04, PNorm = 105.6407, GNorm = 0.0283, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.054194
Epoch 285
Validation binary_cross_entropy = 1.061980
Epoch 286
Validation binary_cross_entropy = 1.071256
Epoch 287
Validation binary_cross_entropy = 1.079639
Epoch 288
Validation binary_cross_entropy = 1.086417
Epoch 289
Loss = 4.1230e-03, PNorm = 105.6583, GNorm = 0.9445, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.102999
Epoch 290
Validation binary_cross_entropy = 1.124645
Epoch 291
Validation binary_cross_entropy = 1.138288
Epoch 292
Validation binary_cross_entropy = 1.142269
Epoch 293
Validation binary_cross_entropy = 1.140951
Epoch 294
Loss = 1.6821e-03, PNorm = 105.6765, GNorm = 0.0330, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.150840
Epoch 295
Validation binary_cross_entropy = 1.158570
Epoch 296
Validation binary_cross_entropy = 1.145649
Epoch 297
Validation binary_cross_entropy = 1.111700
Epoch 298
Validation binary_cross_entropy = 1.090531
Epoch 299
Loss = 9.6282e-04, PNorm = 105.6919, GNorm = 0.0266, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.085311
Model 0 best validation binary_cross_entropy = 0.278861 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.198511
Ensemble test binary_cross_entropy = 0.198511
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.984823
Epoch 1
Validation binary_cross_entropy = 0.299489
Epoch 2
Validation binary_cross_entropy = 1.253395
Epoch 3
Validation binary_cross_entropy = 1.092328
Epoch 4
Loss = 5.9995e-01, PNorm = 96.4351, GNorm = 3.2666, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.311349
Epoch 5
Validation binary_cross_entropy = 1.018172
Epoch 6
Validation binary_cross_entropy = 0.554856
Epoch 7
Validation binary_cross_entropy = 0.405569
Epoch 8
Validation binary_cross_entropy = 0.689790
Epoch 9
Loss = 5.9225e-01, PNorm = 96.7698, GNorm = 3.0522, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.447198
Epoch 10
Validation binary_cross_entropy = 0.364466
Epoch 11
Validation binary_cross_entropy = 0.467733
Epoch 12
Validation binary_cross_entropy = 0.384551
Epoch 13
Validation binary_cross_entropy = 0.387763
Epoch 14
Loss = 3.7463e-01, PNorm = 97.1030, GNorm = 1.9910, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.337910
Epoch 15
Validation binary_cross_entropy = 0.497866
Epoch 16
Validation binary_cross_entropy = 0.297674
Epoch 17
Validation binary_cross_entropy = 0.476969
Epoch 18
Validation binary_cross_entropy = 0.372865
Epoch 19
Loss = 2.2240e-01, PNorm = 97.3515, GNorm = 2.9662, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.334680
Epoch 20
Validation binary_cross_entropy = 0.610183
Epoch 21
Validation binary_cross_entropy = 0.342752
Epoch 22
Validation binary_cross_entropy = 0.791485
Epoch 23
Validation binary_cross_entropy = 0.340196
Epoch 24
Loss = 3.6017e-01, PNorm = 97.5663, GNorm = 9.9440, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.336303
Epoch 25
Validation binary_cross_entropy = 0.804909
Epoch 26
Validation binary_cross_entropy = 0.392604
Epoch 27
Validation binary_cross_entropy = 0.329913
Epoch 28
Validation binary_cross_entropy = 0.412355
Epoch 29
Loss = 2.4019e-01, PNorm = 97.7696, GNorm = 2.6933, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.513859
Epoch 30
Validation binary_cross_entropy = 0.363486
Epoch 31
Validation binary_cross_entropy = 0.407199
Epoch 32
Validation binary_cross_entropy = 0.398691
Epoch 33
Validation binary_cross_entropy = 0.415884
Epoch 34
Loss = 1.2775e-01, PNorm = 97.9744, GNorm = 1.5194, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.392405
Epoch 35
Validation binary_cross_entropy = 0.393979
Epoch 36
Validation binary_cross_entropy = 0.464638
Epoch 37
Validation binary_cross_entropy = 0.448041
Epoch 38
Validation binary_cross_entropy = 0.456037
Epoch 39
Loss = 1.5877e-01, PNorm = 98.1690, GNorm = 8.0821, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.575024
Epoch 40
Validation binary_cross_entropy = 0.583294
Epoch 41
Validation binary_cross_entropy = 0.430840
Epoch 42
Validation binary_cross_entropy = 0.495101
Epoch 43
Validation binary_cross_entropy = 0.505012
Epoch 44
Loss = 2.1101e-01, PNorm = 98.3805, GNorm = 5.0739, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.521922
Epoch 45
Validation binary_cross_entropy = 0.435437
Epoch 46
Validation binary_cross_entropy = 0.441778
Epoch 47
Validation binary_cross_entropy = 0.493902
Epoch 48
Validation binary_cross_entropy = 0.436623
Epoch 49
Loss = 1.7914e-01, PNorm = 98.6129, GNorm = 1.7676, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.456121
Epoch 50
Validation binary_cross_entropy = 0.485380
Epoch 51
Validation binary_cross_entropy = 0.461595
Epoch 52
Validation binary_cross_entropy = 0.474622
Epoch 53
Validation binary_cross_entropy = 0.476969
Epoch 54
Loss = 9.6214e-02, PNorm = 98.8546, GNorm = 0.8944, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.495147
Epoch 55
Validation binary_cross_entropy = 0.573589
Epoch 56
Validation binary_cross_entropy = 0.514103
Epoch 57
Validation binary_cross_entropy = 0.492749
Epoch 58
Validation binary_cross_entropy = 0.504888
Epoch 59
Loss = 1.3622e-01, PNorm = 99.0566, GNorm = 3.5525, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.478515
Epoch 60
Validation binary_cross_entropy = 0.510380
Epoch 61
Validation binary_cross_entropy = 0.492353
Epoch 62
Validation binary_cross_entropy = 0.514662
Epoch 63
Validation binary_cross_entropy = 0.528227
Epoch 64
Loss = 7.7142e-02, PNorm = 99.2516, GNorm = 1.7283, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.544342
Epoch 65
Validation binary_cross_entropy = 0.584718
Epoch 66
Validation binary_cross_entropy = 0.599273
Epoch 67
Validation binary_cross_entropy = 0.594168
Epoch 68
Validation binary_cross_entropy = 0.550905
Epoch 69
Loss = 7.9486e-02, PNorm = 99.4084, GNorm = 1.7839, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.530391
Epoch 70
Validation binary_cross_entropy = 0.525343
Epoch 71
Validation binary_cross_entropy = 0.532661
Epoch 72
Validation binary_cross_entropy = 0.550932
Epoch 73
Validation binary_cross_entropy = 0.581250
Epoch 74
Loss = 3.0385e-02, PNorm = 99.5743, GNorm = 1.0544, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.584844
Epoch 75
Validation binary_cross_entropy = 0.581281
Epoch 76
Validation binary_cross_entropy = 0.586598
Epoch 77
Validation binary_cross_entropy = 0.610853
Epoch 78
Validation binary_cross_entropy = 0.606020
Epoch 79
Loss = 5.0594e-02, PNorm = 99.7354, GNorm = 2.0567, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.598991
Epoch 80
Validation binary_cross_entropy = 0.656749
Epoch 81
Validation binary_cross_entropy = 0.645184
Epoch 82
Validation binary_cross_entropy = 0.640968
Epoch 83
Validation binary_cross_entropy = 0.635511
Epoch 84
Loss = 1.0377e-01, PNorm = 99.8881, GNorm = 2.0069, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.588616
Epoch 85
Validation binary_cross_entropy = 0.604278
Epoch 86
Validation binary_cross_entropy = 0.540547
Epoch 87
Validation binary_cross_entropy = 0.574179
Epoch 88
Validation binary_cross_entropy = 0.599541
Epoch 89
Loss = 9.1103e-02, PNorm = 100.1028, GNorm = 4.4600, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.591650
Epoch 90
Validation binary_cross_entropy = 0.596104
Epoch 91
Validation binary_cross_entropy = 0.566497
Epoch 92
Validation binary_cross_entropy = 0.584935
Epoch 93
Validation binary_cross_entropy = 0.647003
Epoch 94
Loss = 6.2874e-02, PNorm = 100.3293, GNorm = 3.3159, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.607113
Epoch 95
Validation binary_cross_entropy = 0.539708
Epoch 96
Validation binary_cross_entropy = 0.542024
Epoch 97
Validation binary_cross_entropy = 0.607782
Epoch 98
Validation binary_cross_entropy = 0.616577
Epoch 99
Loss = 1.1973e-01, PNorm = 100.5468, GNorm = 3.9622, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.611812
Epoch 100
Validation binary_cross_entropy = 0.614205
Epoch 101
Validation binary_cross_entropy = 0.665141
Epoch 102
Validation binary_cross_entropy = 0.693822
Epoch 103
Validation binary_cross_entropy = 0.613567
Epoch 104
Loss = 1.0968e-01, PNorm = 100.7631, GNorm = 3.1195, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.594014
Epoch 105
Validation binary_cross_entropy = 0.620370
Epoch 106
Validation binary_cross_entropy = 0.623510
Epoch 107
Validation binary_cross_entropy = 0.650722
Epoch 108
Validation binary_cross_entropy = 0.678850
Epoch 109
Loss = 3.6486e-02, PNorm = 100.9341, GNorm = 0.4148, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.704248
Epoch 110
Validation binary_cross_entropy = 0.720702
Epoch 111
Validation binary_cross_entropy = 0.716433
Epoch 112
Validation binary_cross_entropy = 0.723231
Epoch 113
Validation binary_cross_entropy = 0.755344
Epoch 114
Loss = 4.7933e-02, PNorm = 101.0820, GNorm = 1.8047, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.797382
Epoch 115
Validation binary_cross_entropy = 0.773042
Epoch 116
Validation binary_cross_entropy = 0.724504
Epoch 117
Validation binary_cross_entropy = 0.739976
Epoch 118
Validation binary_cross_entropy = 0.774188
Epoch 119
Loss = 1.2191e-02, PNorm = 101.1915, GNorm = 0.1505, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.761614
Epoch 120
Validation binary_cross_entropy = 0.805665
Epoch 121
Validation binary_cross_entropy = 0.850562
Epoch 122
Validation binary_cross_entropy = 0.788585
Epoch 123
Validation binary_cross_entropy = 0.757740
Epoch 124
Loss = 1.5421e-02, PNorm = 101.2925, GNorm = 0.1144, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.766526
Epoch 125
Validation binary_cross_entropy = 0.771806
Epoch 126
Validation binary_cross_entropy = 0.794334
Epoch 127
Validation binary_cross_entropy = 0.790961
Epoch 128
Validation binary_cross_entropy = 0.789814
Epoch 129
Loss = 3.8800e-02, PNorm = 101.4339, GNorm = 0.3110, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.822243
Epoch 130
Validation binary_cross_entropy = 0.841163
Epoch 131
Validation binary_cross_entropy = 0.824246
Epoch 132
Validation binary_cross_entropy = 0.868021
Epoch 133
Validation binary_cross_entropy = 0.937284
Epoch 134
Loss = 3.6635e-02, PNorm = 101.6101, GNorm = 2.1189, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.834422
Epoch 135
Validation binary_cross_entropy = 0.803445
Epoch 136
Validation binary_cross_entropy = 0.904680
Epoch 137
Validation binary_cross_entropy = 0.856892
Epoch 138
Validation binary_cross_entropy = 0.794546
Epoch 139
Loss = 1.2965e-02, PNorm = 101.7371, GNorm = 1.1251, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.807348
Epoch 140
Validation binary_cross_entropy = 0.797928
Epoch 141
Validation binary_cross_entropy = 0.842108
Epoch 142
Validation binary_cross_entropy = 0.879763
Epoch 143
Validation binary_cross_entropy = 0.883016
Epoch 144
Loss = 1.2953e-02, PNorm = 101.8539, GNorm = 1.1376, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.867584
Epoch 145
Validation binary_cross_entropy = 0.960275
Epoch 146
Validation binary_cross_entropy = 1.006315
Epoch 147
Validation binary_cross_entropy = 0.930648
Epoch 148
Validation binary_cross_entropy = 0.892876
Epoch 149
Loss = 6.2485e-02, PNorm = 102.0001, GNorm = 6.0342, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.918300
Epoch 150
Validation binary_cross_entropy = 1.002312
Epoch 151
Validation binary_cross_entropy = 0.887228
Epoch 152
Validation binary_cross_entropy = 0.861561
Epoch 153
Validation binary_cross_entropy = 0.857934
Epoch 154
Loss = 7.0501e-02, PNorm = 102.1907, GNorm = 3.0895, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.932202
Epoch 155
Validation binary_cross_entropy = 1.104882
Epoch 156
Validation binary_cross_entropy = 0.981029
Epoch 157
Validation binary_cross_entropy = 0.897266
Epoch 158
Validation binary_cross_entropy = 0.881388
Epoch 159
Loss = 8.5007e-03, PNorm = 102.4157, GNorm = 0.8535, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.884503
Epoch 160
Validation binary_cross_entropy = 0.921782
Epoch 161
Validation binary_cross_entropy = 0.942235
Epoch 162
Validation binary_cross_entropy = 0.920240
Epoch 163
Validation binary_cross_entropy = 0.929128
Epoch 164
Loss = 3.5094e-02, PNorm = 102.5747, GNorm = 2.6878, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.962980
Epoch 165
Validation binary_cross_entropy = 0.955122
Epoch 166
Validation binary_cross_entropy = 0.921339
Epoch 167
Validation binary_cross_entropy = 0.903879
Epoch 168
Validation binary_cross_entropy = 0.902884
Epoch 169
Loss = 5.0454e-03, PNorm = 102.7182, GNorm = 1.0670, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.912614
Epoch 170
Validation binary_cross_entropy = 0.967597
Epoch 171
Validation binary_cross_entropy = 0.906400
Epoch 172
Validation binary_cross_entropy = 0.950689
Epoch 173
Validation binary_cross_entropy = 0.957794
Epoch 174
Loss = 1.2018e-01, PNorm = 102.8587, GNorm = 5.3856, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.028578
Epoch 175
Validation binary_cross_entropy = 1.030597
Epoch 176
Validation binary_cross_entropy = 0.956460
Epoch 177
Validation binary_cross_entropy = 0.932985
Epoch 178
Validation binary_cross_entropy = 0.915261
Epoch 179
Loss = 2.8913e-02, PNorm = 103.1143, GNorm = 3.3304, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.944194
Epoch 180
Validation binary_cross_entropy = 0.886546
Epoch 181
Validation binary_cross_entropy = 0.843441
Epoch 182
Validation binary_cross_entropy = 0.832416
Epoch 183
Validation binary_cross_entropy = 0.793491
Epoch 184
Loss = 2.7885e-03, PNorm = 103.3304, GNorm = 0.2597, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.786937
Epoch 185
Validation binary_cross_entropy = 0.794369
Epoch 186
Validation binary_cross_entropy = 0.795945
Epoch 187
Validation binary_cross_entropy = 0.792194
Epoch 188
Validation binary_cross_entropy = 0.777155
Epoch 189
Loss = 7.1035e-03, PNorm = 103.4840, GNorm = 0.2762, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.774279
Epoch 190
Validation binary_cross_entropy = 0.798809
Epoch 191
Validation binary_cross_entropy = 0.820825
Epoch 192
Validation binary_cross_entropy = 0.827205
Epoch 193
Validation binary_cross_entropy = 0.867938
Epoch 194
Loss = 1.3659e-02, PNorm = 103.6198, GNorm = 0.7455, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.871025
Epoch 195
Validation binary_cross_entropy = 0.855545
Epoch 196
Validation binary_cross_entropy = 0.875104
Epoch 197
Validation binary_cross_entropy = 0.894871
Epoch 198
Validation binary_cross_entropy = 0.915883
Epoch 199
Loss = 1.0615e-02, PNorm = 103.7213, GNorm = 0.6567, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.918697
Epoch 200
Validation binary_cross_entropy = 0.912919
Epoch 201
Validation binary_cross_entropy = 0.961441
Epoch 202
Validation binary_cross_entropy = 0.950610
Epoch 203
Validation binary_cross_entropy = 0.909735
Epoch 204
Loss = 1.8330e-03, PNorm = 103.8478, GNorm = 0.0415, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.875643
Epoch 205
Validation binary_cross_entropy = 0.859984
Epoch 206
Validation binary_cross_entropy = 0.853062
Epoch 207
Validation binary_cross_entropy = 0.866333
Epoch 208
Validation binary_cross_entropy = 0.897304
Epoch 209
Loss = 8.2840e-03, PNorm = 103.9441, GNorm = 0.9606, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.942564
Epoch 210
Validation binary_cross_entropy = 0.928771
Epoch 211
Validation binary_cross_entropy = 0.896167
Epoch 212
Validation binary_cross_entropy = 0.876666
Epoch 213
Validation binary_cross_entropy = 0.870085
Epoch 214
Loss = 1.9405e-02, PNorm = 104.0207, GNorm = 2.9005, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.871697
Epoch 215
Validation binary_cross_entropy = 0.883038
Epoch 216
Validation binary_cross_entropy = 0.902138
Epoch 217
Validation binary_cross_entropy = 0.908443
Epoch 218
Validation binary_cross_entropy = 0.914068
Epoch 219
Loss = 4.7933e-03, PNorm = 104.0807, GNorm = 0.5725, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.937299
Epoch 220
Validation binary_cross_entropy = 0.962878
Epoch 221
Validation binary_cross_entropy = 0.956948
Epoch 222
Validation binary_cross_entropy = 0.938516
Epoch 223
Validation binary_cross_entropy = 0.940284
Epoch 224
Loss = 1.2407e-03, PNorm = 104.1258, GNorm = 0.1638, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.947710
Epoch 225
Validation binary_cross_entropy = 1.031887
Epoch 226
Validation binary_cross_entropy = 0.997371
Epoch 227
Validation binary_cross_entropy = 0.899972
Epoch 228
Validation binary_cross_entropy = 0.867739
Epoch 229
Loss = 2.6961e-02, PNorm = 104.1908, GNorm = 0.1518, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.876083
Epoch 230
Validation binary_cross_entropy = 0.883643
Epoch 231
Validation binary_cross_entropy = 0.920420
Epoch 232
Validation binary_cross_entropy = 0.916417
Epoch 233
Validation binary_cross_entropy = 0.906595
Epoch 234
Loss = 2.0798e-03, PNorm = 104.3322, GNorm = 0.2632, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.917788
Epoch 235
Validation binary_cross_entropy = 0.951465
Epoch 236
Validation binary_cross_entropy = 0.940493
Epoch 237
Validation binary_cross_entropy = 0.999470
Epoch 238
Validation binary_cross_entropy = 1.069247
Epoch 239
Loss = 2.0278e-02, PNorm = 104.4468, GNorm = 2.5002, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.041028
Epoch 240
Validation binary_cross_entropy = 0.955350
Epoch 241
Validation binary_cross_entropy = 0.946101
Epoch 242
Validation binary_cross_entropy = 0.931471
Epoch 243
Validation binary_cross_entropy = 0.931389
Epoch 244
Loss = 3.0266e-03, PNorm = 104.5521, GNorm = 0.4157, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.975106
Epoch 245
Validation binary_cross_entropy = 0.967559
Epoch 246
Validation binary_cross_entropy = 0.957471
Epoch 247
Validation binary_cross_entropy = 0.976911
Epoch 248
Validation binary_cross_entropy = 1.011444
Epoch 249
Loss = 3.1958e-03, PNorm = 104.6305, GNorm = 0.2976, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.067533
Epoch 250
Validation binary_cross_entropy = 1.098099
Epoch 251
Validation binary_cross_entropy = 1.056426
Epoch 252
Validation binary_cross_entropy = 1.007142
Epoch 253
Validation binary_cross_entropy = 0.999975
Epoch 254
Loss = 1.5215e-03, PNorm = 104.6929, GNorm = 0.0971, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.001057
Epoch 255
Validation binary_cross_entropy = 1.003253
Epoch 256
Validation binary_cross_entropy = 1.007109
Epoch 257
Validation binary_cross_entropy = 1.014089
Epoch 258
Validation binary_cross_entropy = 1.023569
Epoch 259
Loss = 3.8089e-03, PNorm = 104.7482, GNorm = 0.3492, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.037459
Epoch 260
Validation binary_cross_entropy = 1.046298
Epoch 261
Validation binary_cross_entropy = 1.057705
Epoch 262
Validation binary_cross_entropy = 1.071872
Epoch 263
Validation binary_cross_entropy = 1.086036
Epoch 264
Loss = 1.3252e-03, PNorm = 104.8001, GNorm = 0.1121, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.095636
Epoch 265
Validation binary_cross_entropy = 1.098740
Epoch 266
Validation binary_cross_entropy = 1.098485
Epoch 267
Validation binary_cross_entropy = 1.097945
Epoch 268
Validation binary_cross_entropy = 1.100325
Epoch 269
Loss = 6.8014e-04, PNorm = 104.8452, GNorm = 0.0174, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.106510
Epoch 270
Validation binary_cross_entropy = 1.114016
Epoch 271
Validation binary_cross_entropy = 1.120772
Epoch 272
Validation binary_cross_entropy = 1.128105
Epoch 273
Validation binary_cross_entropy = 1.147292
Epoch 274
Loss = 2.4466e-02, PNorm = 104.8779, GNorm = 3.6559, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.112595
Epoch 275
Validation binary_cross_entropy = 1.063213
Epoch 276
Validation binary_cross_entropy = 1.674154
Epoch 277
Validation binary_cross_entropy = 2.213499
Epoch 278
Validation binary_cross_entropy = 2.292937
Epoch 279
Loss = 9.3929e-03, PNorm = 104.9471, GNorm = 1.8685, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 2.270283
Epoch 280
Validation binary_cross_entropy = 0.981930
Epoch 281
Validation binary_cross_entropy = 1.025002
Epoch 282
Validation binary_cross_entropy = 0.937299
Epoch 283
Validation binary_cross_entropy = 0.869426
Epoch 284
Loss = 1.6272e-01, PNorm = 105.0688, GNorm = 3.5876, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.860426
Epoch 285
Validation binary_cross_entropy = 0.893068
Epoch 286
Validation binary_cross_entropy = 0.988917
Epoch 287
Validation binary_cross_entropy = 0.972115
Epoch 288
Validation binary_cross_entropy = 0.865062
Epoch 289
Loss = 2.3320e-02, PNorm = 105.2688, GNorm = 0.0575, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.787330
Epoch 290
Validation binary_cross_entropy = 0.776961
Epoch 291
Validation binary_cross_entropy = 0.792658
Epoch 292
Validation binary_cross_entropy = 0.785676
Epoch 293
Validation binary_cross_entropy = 0.790467
Epoch 294
Loss = 2.1008e-02, PNorm = 105.4210, GNorm = 0.0449, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.826525
Epoch 295
Validation binary_cross_entropy = 0.879638
Epoch 296
Validation binary_cross_entropy = 0.926668
Epoch 297
Validation binary_cross_entropy = 0.933014
Epoch 298
Validation binary_cross_entropy = 0.921104
Epoch 299
Loss = 1.9577e-03, PNorm = 105.5512, GNorm = 0.1588, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.907810
Model 0 best validation binary_cross_entropy = 0.297674 on epoch 16
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.195845
Ensemble test binary_cross_entropy = 0.195845
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.225451
Epoch 1
Validation binary_cross_entropy = 0.295882
Epoch 2
Validation binary_cross_entropy = 1.457155
Epoch 3
Validation binary_cross_entropy = 1.234212
Epoch 4
Loss = 7.4526e-01, PNorm = 96.4354, GNorm = 2.9391, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.303302
Epoch 5
Validation binary_cross_entropy = 0.380694
Epoch 6
Validation binary_cross_entropy = 1.393048
Epoch 7
Validation binary_cross_entropy = 0.495341
Epoch 8
Validation binary_cross_entropy = 0.329228
Epoch 9
Loss = 5.3498e-01, PNorm = 96.7830, GNorm = 1.7081, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.575001
Epoch 10
Validation binary_cross_entropy = 0.740558
Epoch 11
Validation binary_cross_entropy = 0.371947
Epoch 12
Validation binary_cross_entropy = 0.400456
Epoch 13
Validation binary_cross_entropy = 0.757550
Epoch 14
Loss = 4.0183e-01, PNorm = 97.1272, GNorm = 1.8140, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.347492
Epoch 15
Validation binary_cross_entropy = 0.359929
Epoch 16
Validation binary_cross_entropy = 0.574434
Epoch 17
Validation binary_cross_entropy = 0.364325
Epoch 18
Validation binary_cross_entropy = 0.352348
Epoch 19
Loss = 3.1170e-01, PNorm = 97.3752, GNorm = 1.7944, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.480055
Epoch 20
Validation binary_cross_entropy = 0.362994
Epoch 21
Validation binary_cross_entropy = 0.341403
Epoch 22
Validation binary_cross_entropy = 0.368294
Epoch 23
Validation binary_cross_entropy = 0.325948
Epoch 24
Loss = 2.6821e-01, PNorm = 97.6079, GNorm = 5.5394, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.349883
Epoch 25
Validation binary_cross_entropy = 0.647720
Epoch 26
Validation binary_cross_entropy = 0.358216
Epoch 27
Validation binary_cross_entropy = 0.429131
Epoch 28
Validation binary_cross_entropy = 0.361941
Epoch 29
Loss = 2.2415e-01, PNorm = 97.8161, GNorm = 1.8849, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.344025
Epoch 30
Validation binary_cross_entropy = 0.417533
Epoch 31
Validation binary_cross_entropy = 0.504920
Epoch 32
Validation binary_cross_entropy = 0.376723
Epoch 33
Validation binary_cross_entropy = 0.389517
Epoch 34
Loss = 1.6394e-01, PNorm = 98.0280, GNorm = 1.5601, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.414433
Epoch 35
Validation binary_cross_entropy = 0.477719
Epoch 36
Validation binary_cross_entropy = 0.421208
Epoch 37
Validation binary_cross_entropy = 0.452130
Epoch 38
Validation binary_cross_entropy = 0.430398
Epoch 39
Loss = 1.5621e-01, PNorm = 98.2224, GNorm = 2.8989, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.396952
Epoch 40
Validation binary_cross_entropy = 0.368103
Epoch 41
Validation binary_cross_entropy = 0.389070
Epoch 42
Validation binary_cross_entropy = 0.499124
Epoch 43
Validation binary_cross_entropy = 0.458896
Epoch 44
Loss = 8.8372e-02, PNorm = 98.4211, GNorm = 3.5599, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.485345
Epoch 45
Validation binary_cross_entropy = 0.487470
Epoch 46
Validation binary_cross_entropy = 0.461727
Epoch 47
Validation binary_cross_entropy = 0.461415
Epoch 48
Validation binary_cross_entropy = 0.436567
Epoch 49
Loss = 2.4170e-01, PNorm = 98.6159, GNorm = 9.6310, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.444644
Epoch 50
Validation binary_cross_entropy = 0.600716
Epoch 51
Validation binary_cross_entropy = 0.521413
Epoch 52
Validation binary_cross_entropy = 0.515982
Epoch 53
Validation binary_cross_entropy = 0.460906
Epoch 54
Loss = 9.0168e-02, PNorm = 98.8201, GNorm = 1.0405, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.526101
Epoch 55
Validation binary_cross_entropy = 0.460558
Epoch 56
Validation binary_cross_entropy = 0.447887
Epoch 57
Validation binary_cross_entropy = 0.626385
Epoch 58
Validation binary_cross_entropy = 0.560371
Epoch 59
Loss = 2.1086e-01, PNorm = 99.0507, GNorm = 4.2627, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.537451
Epoch 60
Validation binary_cross_entropy = 0.473161
Epoch 61
Validation binary_cross_entropy = 0.486462
Epoch 62
Validation binary_cross_entropy = 0.474958
Epoch 63
Validation binary_cross_entropy = 0.484201
Epoch 64
Loss = 1.2709e-01, PNorm = 99.2935, GNorm = 2.1555, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.474568
Epoch 65
Validation binary_cross_entropy = 0.509852
Epoch 66
Validation binary_cross_entropy = 0.462287
Epoch 67
Validation binary_cross_entropy = 0.519153
Epoch 68
Validation binary_cross_entropy = 0.533976
Epoch 69
Loss = 1.1647e-01, PNorm = 99.5289, GNorm = 3.8078, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.535542
Epoch 70
Validation binary_cross_entropy = 0.546640
Epoch 71
Validation binary_cross_entropy = 0.504695
Epoch 72
Validation binary_cross_entropy = 0.499754
Epoch 73
Validation binary_cross_entropy = 0.467672
Epoch 74
Loss = 7.2587e-02, PNorm = 99.7567, GNorm = 1.1877, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.493035
Epoch 75
Validation binary_cross_entropy = 0.453378
Epoch 76
Validation binary_cross_entropy = 0.507814
Epoch 77
Validation binary_cross_entropy = 0.476482
Epoch 78
Validation binary_cross_entropy = 0.495478
Epoch 79
Loss = 8.2800e-02, PNorm = 99.9649, GNorm = 4.9557, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.545217
Epoch 80
Validation binary_cross_entropy = 0.513858
Epoch 81
Validation binary_cross_entropy = 0.524624
Epoch 82
Validation binary_cross_entropy = 0.538671
Epoch 83
Validation binary_cross_entropy = 0.550936
Epoch 84
Loss = 6.0900e-02, PNorm = 100.1541, GNorm = 2.6086, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.560921
Epoch 85
Validation binary_cross_entropy = 0.567627
Epoch 86
Validation binary_cross_entropy = 0.571240
Epoch 87
Validation binary_cross_entropy = 0.576656
Epoch 88
Validation binary_cross_entropy = 0.594791
Epoch 89
Loss = 3.2934e-02, PNorm = 100.3191, GNorm = 3.0705, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.609192
Epoch 90
Validation binary_cross_entropy = 0.615330
Epoch 91
Validation binary_cross_entropy = 0.641502
Epoch 92
Validation binary_cross_entropy = 0.674006
Epoch 93
Validation binary_cross_entropy = 0.634620
Epoch 94
Loss = 5.0837e-02, PNorm = 100.4626, GNorm = 1.7402, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.644317
Epoch 95
Validation binary_cross_entropy = 0.666162
Epoch 96
Validation binary_cross_entropy = 0.688170
Epoch 97
Validation binary_cross_entropy = 0.744506
Epoch 98
Validation binary_cross_entropy = 0.699484
Epoch 99
Loss = 1.1036e-02, PNorm = 100.5949, GNorm = 0.6998, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.663033
Epoch 100
Validation binary_cross_entropy = 0.661001
Epoch 101
Validation binary_cross_entropy = 0.670659
Epoch 102
Validation binary_cross_entropy = 0.734231
Epoch 103
Validation binary_cross_entropy = 0.717017
Epoch 104
Loss = 4.2209e-02, PNorm = 100.7381, GNorm = 0.8237, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.736139
Epoch 105
Validation binary_cross_entropy = 0.708515
Epoch 106
Validation binary_cross_entropy = 0.805950
Epoch 107
Validation binary_cross_entropy = 0.754504
Epoch 108
Validation binary_cross_entropy = 0.708731
Epoch 109
Loss = 5.8415e-02, PNorm = 100.9414, GNorm = 3.3871, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.700053
Epoch 110
Validation binary_cross_entropy = 0.671010
Epoch 111
Validation binary_cross_entropy = 0.698720
Epoch 112
Validation binary_cross_entropy = 0.677245
Epoch 113
Validation binary_cross_entropy = 0.683714
Epoch 114
Loss = 4.8636e-02, PNorm = 101.1263, GNorm = 3.1730, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.705161
Epoch 115
Validation binary_cross_entropy = 0.770498
Epoch 116
Validation binary_cross_entropy = 0.709097
Epoch 117
Validation binary_cross_entropy = 0.737988
Epoch 118
Validation binary_cross_entropy = 0.722793
Epoch 119
Loss = 3.3231e-02, PNorm = 101.3043, GNorm = 2.1625, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.678167
Epoch 120
Validation binary_cross_entropy = 0.655412
Epoch 121
Validation binary_cross_entropy = 0.676641
Epoch 122
Validation binary_cross_entropy = 0.656700
Epoch 123
Validation binary_cross_entropy = 0.655026
Epoch 124
Loss = 4.1300e-02, PNorm = 101.5137, GNorm = 0.2150, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.661664
Epoch 125
Validation binary_cross_entropy = 0.677476
Epoch 126
Validation binary_cross_entropy = 0.692911
Epoch 127
Validation binary_cross_entropy = 0.694552
Epoch 128
Validation binary_cross_entropy = 0.761641
Epoch 129
Loss = 2.9500e-02, PNorm = 101.7094, GNorm = 2.5938, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.704847
Epoch 130
Validation binary_cross_entropy = 0.663768
Epoch 131
Validation binary_cross_entropy = 0.700979
Epoch 132
Validation binary_cross_entropy = 0.680654
Epoch 133
Validation binary_cross_entropy = 0.686822
Epoch 134
Loss = 1.7493e-02, PNorm = 101.8610, GNorm = 2.0759, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.703499
Epoch 135
Validation binary_cross_entropy = 0.748559
Epoch 136
Validation binary_cross_entropy = 0.862254
Epoch 137
Validation binary_cross_entropy = 0.756659
Epoch 138
Validation binary_cross_entropy = 0.776877
Epoch 139
Loss = 5.7730e-02, PNorm = 102.0183, GNorm = 2.3140, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.743464
Epoch 140
Validation binary_cross_entropy = 0.726037
Epoch 141
Validation binary_cross_entropy = 0.777140
Epoch 142
Validation binary_cross_entropy = 0.770497
Epoch 143
Validation binary_cross_entropy = 0.777823
Epoch 144
Loss = 5.3290e-03, PNorm = 102.1968, GNorm = 0.4622, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.786399
Epoch 145
Validation binary_cross_entropy = 0.811095
Epoch 146
Validation binary_cross_entropy = 0.803125
Epoch 147
Validation binary_cross_entropy = 0.765331
Epoch 148
Validation binary_cross_entropy = 0.748206
Epoch 149
Loss = 8.0839e-03, PNorm = 102.3610, GNorm = 0.7095, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.765133
Epoch 150
Validation binary_cross_entropy = 0.789965
Epoch 151
Validation binary_cross_entropy = 0.800586
Epoch 152
Validation binary_cross_entropy = 0.809370
Epoch 153
Validation binary_cross_entropy = 0.799526
Epoch 154
Loss = 1.3745e-03, PNorm = 102.4783, GNorm = 0.0306, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.772319
Epoch 155
Validation binary_cross_entropy = 0.769950
Epoch 156
Validation binary_cross_entropy = 0.776209
Epoch 157
Validation binary_cross_entropy = 0.790422
Epoch 158
Validation binary_cross_entropy = 0.855691
Epoch 159
Loss = 9.8390e-03, PNorm = 102.5633, GNorm = 0.4375, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.895730
Epoch 160
Validation binary_cross_entropy = 0.839243
Epoch 161
Validation binary_cross_entropy = 0.797938
Epoch 162
Validation binary_cross_entropy = 0.779493
Epoch 163
Validation binary_cross_entropy = 0.824080
Epoch 164
Loss = 2.5755e-03, PNorm = 102.6806, GNorm = 0.1603, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.870354
Epoch 165
Validation binary_cross_entropy = 0.834648
Epoch 166
Validation binary_cross_entropy = 0.797242
Epoch 167
Validation binary_cross_entropy = 0.785302
Epoch 168
Validation binary_cross_entropy = 0.781728
Epoch 169
Loss = 1.2324e-02, PNorm = 102.7992, GNorm = 0.6986, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.816673
Epoch 170
Validation binary_cross_entropy = 0.840758
Epoch 171
Validation binary_cross_entropy = 0.840719
Epoch 172
Validation binary_cross_entropy = 0.838103
Epoch 173
Validation binary_cross_entropy = 0.824246
Epoch 174
Loss = 1.4155e-03, PNorm = 102.9039, GNorm = 0.1813, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.814008
Epoch 175
Validation binary_cross_entropy = 0.814063
Epoch 176
Validation binary_cross_entropy = 0.824362
Epoch 177
Validation binary_cross_entropy = 0.841590
Epoch 178
Validation binary_cross_entropy = 0.858131
Epoch 179
Loss = 2.7002e-02, PNorm = 102.9680, GNorm = 2.8261, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.897554
Epoch 180
Validation binary_cross_entropy = 0.932689
Epoch 181
Validation binary_cross_entropy = 0.905171
Epoch 182
Validation binary_cross_entropy = 0.841622
Epoch 183
Validation binary_cross_entropy = 0.823799
Epoch 184
Loss = 6.0044e-02, PNorm = 103.0538, GNorm = 0.1582, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.911047
Epoch 185
Validation binary_cross_entropy = 1.045284
Epoch 186
Validation binary_cross_entropy = 0.866268
Epoch 187
Validation binary_cross_entropy = 1.504266
Epoch 188
Validation binary_cross_entropy = 0.812337
Epoch 189
Loss = 3.1865e-02, PNorm = 103.2088, GNorm = 4.2912, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.866423
Epoch 190
Validation binary_cross_entropy = 0.813216
Epoch 191
Validation binary_cross_entropy = 0.803317
Epoch 192
Validation binary_cross_entropy = 1.349173
Epoch 193
Validation binary_cross_entropy = 0.875556
Epoch 194
Loss = 4.3326e-02, PNorm = 103.4601, GNorm = 0.8703, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.322870
Epoch 195
Validation binary_cross_entropy = 0.936730
Epoch 196
Validation binary_cross_entropy = 1.050795
Epoch 197
Validation binary_cross_entropy = 1.017118
Epoch 198
Validation binary_cross_entropy = 0.909191
Epoch 199
Loss = 2.3599e-02, PNorm = 103.7597, GNorm = 2.1219, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.848285
Epoch 200
Validation binary_cross_entropy = 0.810770
Epoch 201
Validation binary_cross_entropy = 0.790969
Epoch 202
Validation binary_cross_entropy = 0.829350
Epoch 203
Validation binary_cross_entropy = 0.790450
Epoch 204
Loss = 3.4049e-03, PNorm = 104.0209, GNorm = 0.1313, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.776837
Epoch 205
Validation binary_cross_entropy = 0.792193
Epoch 206
Validation binary_cross_entropy = 0.817738
Epoch 207
Validation binary_cross_entropy = 0.885391
Epoch 208
Validation binary_cross_entropy = 0.861113
Epoch 209
Loss = 4.7077e-02, PNorm = 104.2105, GNorm = 2.0468, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.836811
Epoch 210
Validation binary_cross_entropy = 0.792733
Epoch 211
Validation binary_cross_entropy = 0.786904
Epoch 212
Validation binary_cross_entropy = 0.794608
Epoch 213
Validation binary_cross_entropy = 0.833817
Epoch 214
Loss = 3.5047e-03, PNorm = 104.3878, GNorm = 0.1541, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.869589
Epoch 215
Validation binary_cross_entropy = 0.889346
Epoch 216
Validation binary_cross_entropy = 0.880863
Epoch 217
Validation binary_cross_entropy = 0.887820
Epoch 218
Validation binary_cross_entropy = 0.914487
Epoch 219
Loss = 3.7926e-03, PNorm = 104.5232, GNorm = 0.4870, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.947311
Epoch 220
Validation binary_cross_entropy = 1.022137
Epoch 221
Validation binary_cross_entropy = 0.990011
Epoch 222
Validation binary_cross_entropy = 0.914984
Epoch 223
Validation binary_cross_entropy = 0.905610
Epoch 224
Loss = 2.8894e-02, PNorm = 104.6217, GNorm = 2.7014, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.923005
Epoch 225
Validation binary_cross_entropy = 0.913927
Epoch 226
Validation binary_cross_entropy = 1.003970
Epoch 227
Validation binary_cross_entropy = 1.038910
Epoch 228
Validation binary_cross_entropy = 0.993762
Epoch 229
Loss = 1.0580e-01, PNorm = 104.7399, GNorm = 0.0188, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.048737
Epoch 230
Validation binary_cross_entropy = 1.563678
Epoch 231
Validation binary_cross_entropy = 0.913192
Epoch 232
Validation binary_cross_entropy = 0.842820
Epoch 233
Validation binary_cross_entropy = 0.825528
Epoch 234
Loss = 6.4980e-03, PNorm = 104.9472, GNorm = 0.6405, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.833268
Epoch 235
Validation binary_cross_entropy = 0.818299
Epoch 236
Validation binary_cross_entropy = 0.784476
Epoch 237
Validation binary_cross_entropy = 0.772258
Epoch 238
Validation binary_cross_entropy = 0.773703
Epoch 239
Loss = 3.8551e-03, PNorm = 105.1301, GNorm = 0.2315, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.763242
Epoch 240
Validation binary_cross_entropy = 0.727853
Epoch 241
Validation binary_cross_entropy = 0.725655
Epoch 242
Validation binary_cross_entropy = 0.736995
Epoch 243
Validation binary_cross_entropy = 0.741274
Epoch 244
Loss = 1.7139e-03, PNorm = 105.2833, GNorm = 0.1311, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.752409
Epoch 245
Validation binary_cross_entropy = 0.768182
Epoch 246
Validation binary_cross_entropy = 0.790216
Epoch 247
Validation binary_cross_entropy = 0.810403
Epoch 248
Validation binary_cross_entropy = 0.815441
Epoch 249
Loss = 9.5160e-04, PNorm = 105.3690, GNorm = 0.0157, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.814706
Epoch 250
Validation binary_cross_entropy = 0.812775
Epoch 251
Validation binary_cross_entropy = 0.821864
Epoch 252
Validation binary_cross_entropy = 0.829965
Epoch 253
Validation binary_cross_entropy = 0.840752
Epoch 254
Loss = 8.5397e-03, PNorm = 105.4242, GNorm = 1.2471, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.842963
Epoch 255
Validation binary_cross_entropy = 0.829743
Epoch 256
Validation binary_cross_entropy = 0.819947
Epoch 257
Validation binary_cross_entropy = 0.851968
Epoch 258
Validation binary_cross_entropy = 0.877813
Epoch 259
Loss = 4.4230e-03, PNorm = 105.4846, GNorm = 0.0679, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.879983
Epoch 260
Validation binary_cross_entropy = 0.880698
Epoch 261
Validation binary_cross_entropy = 0.885396
Epoch 262
Validation binary_cross_entropy = 0.879128
Epoch 263
Validation binary_cross_entropy = 0.881126
Epoch 264
Loss = 1.8626e-03, PNorm = 105.5253, GNorm = 0.0864, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.885034
Epoch 265
Validation binary_cross_entropy = 0.894779
Epoch 266
Validation binary_cross_entropy = 0.914583
Epoch 267
Validation binary_cross_entropy = 0.928730
Epoch 268
Validation binary_cross_entropy = 0.933992
Epoch 269
Loss = 6.2804e-04, PNorm = 105.5489, GNorm = 0.0586, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.939473
Epoch 270
Validation binary_cross_entropy = 0.947340
Epoch 271
Validation binary_cross_entropy = 0.946788
Epoch 272
Validation binary_cross_entropy = 0.945744
Epoch 273
Validation binary_cross_entropy = 0.933490
Epoch 274
Loss = 4.0250e-04, PNorm = 105.5705, GNorm = 0.0598, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.920673
Epoch 275
Validation binary_cross_entropy = 0.922899
Epoch 276
Validation binary_cross_entropy = 0.939713
Epoch 277
Validation binary_cross_entropy = 0.954574
Epoch 278
Validation binary_cross_entropy = 0.969265
Epoch 279
Loss = 7.7491e-04, PNorm = 105.5944, GNorm = 0.0850, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.979317
Epoch 280
Validation binary_cross_entropy = 0.983505
Epoch 281
Validation binary_cross_entropy = 1.012901
Epoch 282
Validation binary_cross_entropy = 1.036788
Epoch 283
Validation binary_cross_entropy = 1.034971
Epoch 284
Loss = 4.8197e-04, PNorm = 105.6206, GNorm = 0.0198, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.014808
Epoch 285
Validation binary_cross_entropy = 0.991618
Epoch 286
Validation binary_cross_entropy = 0.975533
Epoch 287
Validation binary_cross_entropy = 0.959625
Epoch 288
Validation binary_cross_entropy = 0.959441
Epoch 289
Loss = 5.8020e-05, PNorm = 105.6422, GNorm = 0.0036, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.959389
Epoch 290
Validation binary_cross_entropy = 0.963007
Epoch 291
Validation binary_cross_entropy = 0.970604
Epoch 292
Validation binary_cross_entropy = 0.984389
Epoch 293
Validation binary_cross_entropy = 0.997193
Epoch 294
Loss = 5.9320e-04, PNorm = 105.6521, GNorm = 0.1119, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.008054
Epoch 295
Validation binary_cross_entropy = 1.019916
Epoch 296
Validation binary_cross_entropy = 1.020309
Epoch 297
Validation binary_cross_entropy = 1.006679
Epoch 298
Validation binary_cross_entropy = 0.996909
Epoch 299
Loss = 7.1811e-04, PNorm = 105.6713, GNorm = 0.1425, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.992835
Model 0 best validation binary_cross_entropy = 0.295882 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.197648
Ensemble test binary_cross_entropy = 0.197648
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.245632
Epoch 1
Validation binary_cross_entropy = 0.295420
Epoch 2
Validation binary_cross_entropy = 1.894859
Epoch 3
Validation binary_cross_entropy = 0.762522
Epoch 4
Loss = 6.2391e-01, PNorm = 96.4335, GNorm = 4.1164, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.312397
Epoch 5
Validation binary_cross_entropy = 0.782820
Epoch 6
Validation binary_cross_entropy = 1.429839
Epoch 7
Validation binary_cross_entropy = 0.289436
Epoch 8
Validation binary_cross_entropy = 0.452709
Epoch 9
Loss = 5.4112e-01, PNorm = 96.7713, GNorm = 4.7019, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.781250
Epoch 10
Validation binary_cross_entropy = 0.341912
Epoch 11
Validation binary_cross_entropy = 0.338396
Epoch 12
Validation binary_cross_entropy = 0.576503
Epoch 13
Validation binary_cross_entropy = 0.474862
Epoch 14
Loss = 4.1327e-01, PNorm = 97.0987, GNorm = 3.4033, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.309565
Epoch 15
Validation binary_cross_entropy = 0.484165
Epoch 16
Validation binary_cross_entropy = 0.453149
Epoch 17
Validation binary_cross_entropy = 0.359099
Epoch 18
Validation binary_cross_entropy = 0.489181
Epoch 19
Loss = 3.0545e-01, PNorm = 97.3488, GNorm = 2.3860, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.460671
Epoch 20
Validation binary_cross_entropy = 0.485912
Epoch 21
Validation binary_cross_entropy = 0.423247
Epoch 22
Validation binary_cross_entropy = 0.351191
Epoch 23
Validation binary_cross_entropy = 0.577246
Epoch 24
Loss = 2.6451e-01, PNorm = 97.5635, GNorm = 1.1765, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.332281
Epoch 25
Validation binary_cross_entropy = 0.468891
Epoch 26
Validation binary_cross_entropy = 0.704132
Epoch 27
Validation binary_cross_entropy = 0.427990
Epoch 28
Validation binary_cross_entropy = 0.460135
Epoch 29
Loss = 2.3785e-01, PNorm = 97.7894, GNorm = 2.7693, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.510609
Epoch 30
Validation binary_cross_entropy = 0.425934
Epoch 31
Validation binary_cross_entropy = 0.401530
Epoch 32
Validation binary_cross_entropy = 0.603909
Epoch 33
Validation binary_cross_entropy = 0.383872
Epoch 34
Loss = 1.8345e-01, PNorm = 98.0052, GNorm = 2.9964, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.393666
Epoch 35
Validation binary_cross_entropy = 0.485031
Epoch 36
Validation binary_cross_entropy = 0.421879
Epoch 37
Validation binary_cross_entropy = 0.443329
Epoch 38
Validation binary_cross_entropy = 0.538696
Epoch 39
Loss = 2.4816e-01, PNorm = 98.2430, GNorm = 3.8948, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.455409
Epoch 40
Validation binary_cross_entropy = 0.405041
Epoch 41
Validation binary_cross_entropy = 0.456152
Epoch 42
Validation binary_cross_entropy = 0.520614
Epoch 43
Validation binary_cross_entropy = 0.470535
Epoch 44
Loss = 1.9155e-01, PNorm = 98.4777, GNorm = 1.2970, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.483504
Epoch 45
Validation binary_cross_entropy = 0.557234
Epoch 46
Validation binary_cross_entropy = 0.498316
Epoch 47
Validation binary_cross_entropy = 0.522086
Epoch 48
Validation binary_cross_entropy = 0.495875
Epoch 49
Loss = 4.6857e-02, PNorm = 98.7330, GNorm = 1.1137, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.458845
Epoch 50
Validation binary_cross_entropy = 0.456232
Epoch 51
Validation binary_cross_entropy = 0.466237
Epoch 52
Validation binary_cross_entropy = 0.467811
Epoch 53
Validation binary_cross_entropy = 0.473362
Epoch 54
Loss = 1.1153e-01, PNorm = 98.9760, GNorm = 3.9018, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.489086
Epoch 55
Validation binary_cross_entropy = 0.554721
Epoch 56
Validation binary_cross_entropy = 0.501323
Epoch 57
Validation binary_cross_entropy = 0.446768
Epoch 58
Validation binary_cross_entropy = 0.444156
Epoch 59
Loss = 7.1445e-02, PNorm = 99.1726, GNorm = 0.8831, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.433395
Epoch 60
Validation binary_cross_entropy = 0.432240
Epoch 61
Validation binary_cross_entropy = 0.428679
Epoch 62
Validation binary_cross_entropy = 0.447891
Epoch 63
Validation binary_cross_entropy = 0.450515
Epoch 64
Loss = 5.8739e-02, PNorm = 99.3671, GNorm = 0.5802, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.491741
Epoch 65
Validation binary_cross_entropy = 0.511930
Epoch 66
Validation binary_cross_entropy = 0.521677
Epoch 67
Validation binary_cross_entropy = 0.530602
Epoch 68
Validation binary_cross_entropy = 0.535549
Epoch 69
Loss = 5.3725e-02, PNorm = 99.5667, GNorm = 0.9697, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.586230
Epoch 70
Validation binary_cross_entropy = 0.546729
Epoch 71
Validation binary_cross_entropy = 0.541155
Epoch 72
Validation binary_cross_entropy = 0.598674
Epoch 73
Validation binary_cross_entropy = 0.550979
Epoch 74
Loss = 6.5234e-02, PNorm = 99.7287, GNorm = 1.7493, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.547077
Epoch 75
Validation binary_cross_entropy = 0.566835
Epoch 76
Validation binary_cross_entropy = 0.603061
Epoch 77
Validation binary_cross_entropy = 0.638581
Epoch 78
Validation binary_cross_entropy = 0.596405
Epoch 79
Loss = 1.2588e-02, PNorm = 99.8876, GNorm = 1.3571, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.673969
Epoch 80
Validation binary_cross_entropy = 0.617642
Epoch 81
Validation binary_cross_entropy = 0.624741
Epoch 82
Validation binary_cross_entropy = 0.668132
Epoch 83
Validation binary_cross_entropy = 0.645997
Epoch 84
Loss = 3.8695e-02, PNorm = 100.0237, GNorm = 1.2017, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.590691
Epoch 85
Validation binary_cross_entropy = 0.601111
Epoch 86
Validation binary_cross_entropy = 0.633642
Epoch 87
Validation binary_cross_entropy = 0.639708
Epoch 88
Validation binary_cross_entropy = 0.627020
Epoch 89
Loss = 1.1924e-01, PNorm = 100.1785, GNorm = 2.7367, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.652259
Epoch 90
Validation binary_cross_entropy = 0.597511
Epoch 91
Validation binary_cross_entropy = 0.577609
Epoch 92
Validation binary_cross_entropy = 0.582128
Epoch 93
Validation binary_cross_entropy = 0.576720
Epoch 94
Loss = 7.4744e-02, PNorm = 100.3313, GNorm = 1.3240, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.599906
Epoch 95
Validation binary_cross_entropy = 0.631789
Epoch 96
Validation binary_cross_entropy = 0.620419
Epoch 97
Validation binary_cross_entropy = 0.628271
Epoch 98
Validation binary_cross_entropy = 0.600494
Epoch 99
Loss = 1.8220e-02, PNorm = 100.5280, GNorm = 0.7107, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.598083
Epoch 100
Validation binary_cross_entropy = 0.641940
Epoch 101
Validation binary_cross_entropy = 0.678818
Epoch 102
Validation binary_cross_entropy = 0.652439
Epoch 103
Validation binary_cross_entropy = 0.623284
Epoch 104
Loss = 1.5984e-01, PNorm = 100.7275, GNorm = 5.0760, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.625165
Epoch 105
Validation binary_cross_entropy = 0.695122
Epoch 106
Validation binary_cross_entropy = 0.614803
Epoch 107
Validation binary_cross_entropy = 0.649872
Epoch 108
Validation binary_cross_entropy = 0.797474
Epoch 109
Loss = 3.2129e-02, PNorm = 101.0014, GNorm = 1.6582, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.388577
Epoch 110
Validation binary_cross_entropy = 0.801089
Epoch 111
Validation binary_cross_entropy = 0.720293
Epoch 112
Validation binary_cross_entropy = 0.814961
Epoch 113
Validation binary_cross_entropy = 0.771811
Epoch 114
Loss = 1.1508e-01, PNorm = 101.3141, GNorm = 1.2134, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.675254
Epoch 115
Validation binary_cross_entropy = 0.714439
Epoch 116
Validation binary_cross_entropy = 0.680254
Epoch 117
Validation binary_cross_entropy = 0.632333
Epoch 118
Validation binary_cross_entropy = 0.640343
Epoch 119
Loss = 3.7139e-02, PNorm = 101.6867, GNorm = 2.1293, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.710884
Epoch 120
Validation binary_cross_entropy = 0.654539
Epoch 121
Validation binary_cross_entropy = 0.707574
Epoch 122
Validation binary_cross_entropy = 0.730656
Epoch 123
Validation binary_cross_entropy = 0.687674
Epoch 124
Loss = 2.4571e-02, PNorm = 102.0101, GNorm = 1.2449, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.685830
Epoch 125
Validation binary_cross_entropy = 0.721861
Epoch 126
Validation binary_cross_entropy = 0.763411
Epoch 127
Validation binary_cross_entropy = 0.805071
Epoch 128
Validation binary_cross_entropy = 0.833413
Epoch 129
Loss = 8.0427e-02, PNorm = 102.2975, GNorm = 2.1648, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.861379
Epoch 130
Validation binary_cross_entropy = 0.875323
Epoch 131
Validation binary_cross_entropy = 0.771586
Epoch 132
Validation binary_cross_entropy = 0.746825
Epoch 133
Validation binary_cross_entropy = 0.791405
Epoch 134
Loss = 6.2720e-02, PNorm = 102.5832, GNorm = 2.9127, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.736096
Epoch 135
Validation binary_cross_entropy = 0.712073
Epoch 136
Validation binary_cross_entropy = 0.782183
Epoch 137
Validation binary_cross_entropy = 0.802408
Epoch 138
Validation binary_cross_entropy = 0.781528
Epoch 139
Loss = 1.5482e-02, PNorm = 102.7744, GNorm = 1.4818, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.770597
Epoch 140
Validation binary_cross_entropy = 0.791870
Epoch 141
Validation binary_cross_entropy = 0.834716
Epoch 142
Validation binary_cross_entropy = 0.838458
Epoch 143
Validation binary_cross_entropy = 0.731131
Epoch 144
Loss = 2.0124e-02, PNorm = 102.9549, GNorm = 0.4635, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.706822
Epoch 145
Validation binary_cross_entropy = 0.713077
Epoch 146
Validation binary_cross_entropy = 0.733115
Epoch 147
Validation binary_cross_entropy = 0.718470
Epoch 148
Validation binary_cross_entropy = 0.720517
Epoch 149
Loss = 6.7384e-03, PNorm = 103.1085, GNorm = 0.2316, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.734506
Epoch 150
Validation binary_cross_entropy = 0.747104
Epoch 151
Validation binary_cross_entropy = 0.746093
Epoch 152
Validation binary_cross_entropy = 0.763663
Epoch 153
Validation binary_cross_entropy = 0.783855
Epoch 154
Loss = 4.2838e-03, PNorm = 103.2275, GNorm = 0.2352, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.791197
Epoch 155
Validation binary_cross_entropy = 0.785277
Epoch 156
Validation binary_cross_entropy = 0.776827
Epoch 157
Validation binary_cross_entropy = 0.777336
Epoch 158
Validation binary_cross_entropy = 0.806289
Epoch 159
Loss = 4.3539e-03, PNorm = 103.3157, GNorm = 0.4864, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.831369
Epoch 160
Validation binary_cross_entropy = 0.854725
Epoch 161
Validation binary_cross_entropy = 0.854453
Epoch 162
Validation binary_cross_entropy = 0.851427
Epoch 163
Validation binary_cross_entropy = 0.860899
Epoch 164
Loss = 1.0014e-02, PNorm = 103.3870, GNorm = 1.2408, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.891429
Epoch 165
Validation binary_cross_entropy = 0.914511
Epoch 166
Validation binary_cross_entropy = 0.898958
Epoch 167
Validation binary_cross_entropy = 0.869490
Epoch 168
Validation binary_cross_entropy = 0.860473
Epoch 169
Loss = 6.5676e-04, PNorm = 103.4510, GNorm = 0.0539, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.860582
Epoch 170
Validation binary_cross_entropy = 0.864377
Epoch 171
Validation binary_cross_entropy = 0.854700
Epoch 172
Validation binary_cross_entropy = 0.873637
Epoch 173
Validation binary_cross_entropy = 0.916357
Epoch 174
Loss = 5.9490e-03, PNorm = 103.5085, GNorm = 0.7004, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.927556
Epoch 175
Validation binary_cross_entropy = 0.914065
Epoch 176
Validation binary_cross_entropy = 0.891642
Epoch 177
Validation binary_cross_entropy = 0.899914
Epoch 178
Validation binary_cross_entropy = 0.925502
Epoch 179
Loss = 1.8520e-02, PNorm = 103.5915, GNorm = 1.7058, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.925889
Epoch 180
Validation binary_cross_entropy = 0.970488
Epoch 181
Validation binary_cross_entropy = 1.071426
Epoch 182
Validation binary_cross_entropy = 0.972330
Epoch 183
Validation binary_cross_entropy = 0.914108
Epoch 184
Loss = 1.2574e-02, PNorm = 103.6847, GNorm = 2.3611, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.914591
Epoch 185
Validation binary_cross_entropy = 0.903278
Epoch 186
Validation binary_cross_entropy = 1.139295
Epoch 187
Validation binary_cross_entropy = 0.844244
Epoch 188
Validation binary_cross_entropy = 1.641188
Epoch 189
Loss = 8.8780e-02, PNorm = 103.8547, GNorm = 0.3636, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.804884
Epoch 190
Validation binary_cross_entropy = 0.875859
Epoch 191
Validation binary_cross_entropy = 0.849512
Epoch 192
Validation binary_cross_entropy = 0.879754
Epoch 193
Validation binary_cross_entropy = 0.902767
Epoch 194
Loss = 6.2324e-03, PNorm = 104.0935, GNorm = 0.5600, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.914268
Epoch 195
Validation binary_cross_entropy = 0.964210
Epoch 196
Validation binary_cross_entropy = 0.893190
Epoch 197
Validation binary_cross_entropy = 0.849124
Epoch 198
Validation binary_cross_entropy = 0.843150
Epoch 199
Loss = 4.7703e-02, PNorm = 104.3060, GNorm = 4.4961, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.804132
Epoch 200
Validation binary_cross_entropy = 0.848024
Epoch 201
Validation binary_cross_entropy = 0.828506
Epoch 202
Validation binary_cross_entropy = 0.750658
Epoch 203
Validation binary_cross_entropy = 0.734481
Epoch 204
Loss = 4.9613e-02, PNorm = 104.4849, GNorm = 2.9488, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.724311
Epoch 205
Validation binary_cross_entropy = 0.747029
Epoch 206
Validation binary_cross_entropy = 0.839772
Epoch 207
Validation binary_cross_entropy = 0.816637
Epoch 208
Validation binary_cross_entropy = 0.727553
Epoch 209
Loss = 4.5065e-03, PNorm = 104.6333, GNorm = 0.5952, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.696405
Epoch 210
Validation binary_cross_entropy = 0.702695
Epoch 211
Validation binary_cross_entropy = 0.729940
Epoch 212
Validation binary_cross_entropy = 0.802805
Epoch 213
Validation binary_cross_entropy = 0.838135
Epoch 214
Loss = 3.0871e-02, PNorm = 104.7324, GNorm = 2.2242, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.786146
Epoch 215
Validation binary_cross_entropy = 0.755430
Epoch 216
Validation binary_cross_entropy = 0.759675
Epoch 217
Validation binary_cross_entropy = 0.772973
Epoch 218
Validation binary_cross_entropy = 0.789120
Epoch 219
Loss = 1.9554e-03, PNorm = 104.8249, GNorm = 0.1232, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.831846
Epoch 220
Validation binary_cross_entropy = 0.860323
Epoch 221
Validation binary_cross_entropy = 0.868257
Epoch 222
Validation binary_cross_entropy = 0.835791
Epoch 223
Validation binary_cross_entropy = 0.825512
Epoch 224
Loss = 2.3193e-03, PNorm = 104.9056, GNorm = 0.3523, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.827555
Epoch 225
Validation binary_cross_entropy = 0.835296
Epoch 226
Validation binary_cross_entropy = 0.841674
Epoch 227
Validation binary_cross_entropy = 0.853670
Epoch 228
Validation binary_cross_entropy = 0.870258
Epoch 229
Loss = 3.4213e-03, PNorm = 104.9651, GNorm = 0.0334, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.884123
Epoch 230
Validation binary_cross_entropy = 0.903531
Epoch 231
Validation binary_cross_entropy = 0.921755
Epoch 232
Validation binary_cross_entropy = 0.926128
Epoch 233
Validation binary_cross_entropy = 0.924867
Epoch 234
Loss = 2.4392e-03, PNorm = 105.0072, GNorm = 0.0898, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.921102
Epoch 235
Validation binary_cross_entropy = 0.917191
Epoch 236
Validation binary_cross_entropy = 0.912436
Epoch 237
Validation binary_cross_entropy = 0.907793
Epoch 238
Validation binary_cross_entropy = 0.912291
Epoch 239
Loss = 8.7168e-04, PNorm = 105.0376, GNorm = 0.0312, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.917268
Epoch 240
Validation binary_cross_entropy = 0.931488
Epoch 241
Validation binary_cross_entropy = 0.954262
Epoch 242
Validation binary_cross_entropy = 0.969781
Epoch 243
Validation binary_cross_entropy = 0.977872
Epoch 244
Loss = 1.3583e-03, PNorm = 105.0683, GNorm = 0.1087, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.979618
Epoch 245
Validation binary_cross_entropy = 0.977562
Epoch 246
Validation binary_cross_entropy = 0.975643
Epoch 247
Validation binary_cross_entropy = 0.974687
Epoch 248
Validation binary_cross_entropy = 0.977070
Epoch 249
Loss = 8.0123e-04, PNorm = 105.0935, GNorm = 0.0857, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.982530
Epoch 250
Validation binary_cross_entropy = 0.991790
Epoch 251
Validation binary_cross_entropy = 1.010509
Epoch 252
Validation binary_cross_entropy = 1.036970
Epoch 253
Validation binary_cross_entropy = 1.059664
Epoch 254
Loss = 9.3122e-04, PNorm = 105.1081, GNorm = 0.0481, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.071264
Epoch 255
Validation binary_cross_entropy = 1.079461
Epoch 256
Validation binary_cross_entropy = 1.082857
Epoch 257
Validation binary_cross_entropy = 1.093772
Epoch 258
Validation binary_cross_entropy = 1.100759
Epoch 259
Loss = 3.6983e-02, PNorm = 105.1376, GNorm = 0.0242, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.004611
Epoch 260
Validation binary_cross_entropy = 0.998213
Epoch 261
Validation binary_cross_entropy = 1.005280
Epoch 262
Validation binary_cross_entropy = 0.991880
Epoch 263
Validation binary_cross_entropy = 0.984426
Epoch 264
Loss = 7.5815e-04, PNorm = 105.1944, GNorm = 0.0189, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.077874
Epoch 265
Validation binary_cross_entropy = 1.124297
Epoch 266
Validation binary_cross_entropy = 1.102769
Epoch 267
Validation binary_cross_entropy = 1.085205
Epoch 268
Validation binary_cross_entropy = 1.060920
Epoch 269
Loss = 2.8217e-03, PNorm = 105.2898, GNorm = 0.2493, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.047781
Epoch 270
Validation binary_cross_entropy = 1.051456
Epoch 271
Validation binary_cross_entropy = 1.062112
Epoch 272
Validation binary_cross_entropy = 1.070093
Epoch 273
Validation binary_cross_entropy = 1.087402
Epoch 274
Loss = 4.5002e-03, PNorm = 105.3724, GNorm = 0.2467, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.100821
Epoch 275
Validation binary_cross_entropy = 1.108664
Epoch 276
Validation binary_cross_entropy = 1.111887
Epoch 277
Validation binary_cross_entropy = 1.087508
Epoch 278
Validation binary_cross_entropy = 1.081188
Epoch 279
Loss = 1.1402e-02, PNorm = 105.4426, GNorm = 1.8339, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.056022
Epoch 280
Validation binary_cross_entropy = 1.053397
Epoch 281
Validation binary_cross_entropy = 1.174499
Epoch 282
Validation binary_cross_entropy = 1.236809
Epoch 283
Validation binary_cross_entropy = 1.033733
Epoch 284
Loss = 6.6815e-03, PNorm = 105.6086, GNorm = 1.4677, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.938752
Epoch 285
Validation binary_cross_entropy = 1.453215
Epoch 286
Validation binary_cross_entropy = 0.908682
Epoch 287
Validation binary_cross_entropy = 0.935128
Epoch 288
Validation binary_cross_entropy = 0.948283
Epoch 289
Loss = 1.9849e-03, PNorm = 105.7333, GNorm = 0.3022, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.968694
Epoch 290
Validation binary_cross_entropy = 0.965527
Epoch 291
Validation binary_cross_entropy = 0.947526
Epoch 292
Validation binary_cross_entropy = 0.902950
Epoch 293
Validation binary_cross_entropy = 0.877613
Epoch 294
Loss = 2.6014e-02, PNorm = 105.8303, GNorm = 0.9878, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.975628
Epoch 295
Validation binary_cross_entropy = 1.030628
Epoch 296
Validation binary_cross_entropy = 1.034226
Epoch 297
Validation binary_cross_entropy = 0.996728
Epoch 298
Validation binary_cross_entropy = 1.477542
Epoch 299
Loss = 1.2620e-03, PNorm = 105.9433, GNorm = 0.1122, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.435138
Model 0 best validation binary_cross_entropy = 0.289436 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.220936
Ensemble test binary_cross_entropy = 0.220936
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.102035
Epoch 1
Validation binary_cross_entropy = 0.313893
Epoch 2
Validation binary_cross_entropy = 1.536830
Epoch 3
Validation binary_cross_entropy = 0.592376
Epoch 4
Loss = 6.4594e-01, PNorm = 96.4303, GNorm = 8.2283, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.474685
Epoch 5
Validation binary_cross_entropy = 1.372457
Epoch 6
Validation binary_cross_entropy = 0.347555
Epoch 7
Validation binary_cross_entropy = 0.379125
Epoch 8
Validation binary_cross_entropy = 0.761961
Epoch 9
Loss = 4.0330e-01, PNorm = 96.7507, GNorm = 2.5946, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.313624
Epoch 10
Validation binary_cross_entropy = 0.421175
Epoch 11
Validation binary_cross_entropy = 0.534624
Epoch 12
Validation binary_cross_entropy = 0.314692
Epoch 13
Validation binary_cross_entropy = 0.396397
Epoch 14
Loss = 3.9113e-01, PNorm = 97.0630, GNorm = 2.8301, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.403507
Epoch 15
Validation binary_cross_entropy = 0.288279
Epoch 16
Validation binary_cross_entropy = 0.497700
Epoch 17
Validation binary_cross_entropy = 0.346119
Epoch 18
Validation binary_cross_entropy = 0.384938
Epoch 19
Loss = 2.6006e-01, PNorm = 97.3015, GNorm = 1.0008, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.857604
Epoch 20
Validation binary_cross_entropy = 0.372626
Epoch 21
Validation binary_cross_entropy = 0.498998
Epoch 22
Validation binary_cross_entropy = 0.714172
Epoch 23
Validation binary_cross_entropy = 0.340989
Epoch 24
Loss = 3.4003e-01, PNorm = 97.5108, GNorm = 6.9754, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.358620
Epoch 25
Validation binary_cross_entropy = 0.658433
Epoch 26
Validation binary_cross_entropy = 0.384395
Epoch 27
Validation binary_cross_entropy = 0.332526
Epoch 28
Validation binary_cross_entropy = 0.492998
Epoch 29
Loss = 2.0875e-01, PNorm = 97.7207, GNorm = 3.5815, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.484915
Epoch 30
Validation binary_cross_entropy = 0.367547
Epoch 31
Validation binary_cross_entropy = 0.397958
Epoch 32
Validation binary_cross_entropy = 0.457661
Epoch 33
Validation binary_cross_entropy = 0.361469
Epoch 34
Loss = 2.3487e-01, PNorm = 97.9163, GNorm = 2.9088, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.350638
Epoch 35
Validation binary_cross_entropy = 0.386303
Epoch 36
Validation binary_cross_entropy = 0.420116
Epoch 37
Validation binary_cross_entropy = 0.428019
Epoch 38
Validation binary_cross_entropy = 0.414728
Epoch 39
Loss = 1.4547e-01, PNorm = 98.1221, GNorm = 2.1904, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.428437
Epoch 40
Validation binary_cross_entropy = 0.442784
Epoch 41
Validation binary_cross_entropy = 0.470375
Epoch 42
Validation binary_cross_entropy = 0.430093
Epoch 43
Validation binary_cross_entropy = 0.457300
Epoch 44
Loss = 7.9328e-02, PNorm = 98.3219, GNorm = 2.7065, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.604591
Epoch 45
Validation binary_cross_entropy = 0.499502
Epoch 46
Validation binary_cross_entropy = 0.497887
Epoch 47
Validation binary_cross_entropy = 0.500405
Epoch 48
Validation binary_cross_entropy = 0.485878
Epoch 49
Loss = 1.4725e-01, PNorm = 98.5346, GNorm = 5.5724, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.489211
Epoch 50
Validation binary_cross_entropy = 0.622772
Epoch 51
Validation binary_cross_entropy = 0.527591
Epoch 52
Validation binary_cross_entropy = 0.491637
Epoch 53
Validation binary_cross_entropy = 0.539912
Epoch 54
Loss = 1.0519e-01, PNorm = 98.7899, GNorm = 3.2932, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.599946
Epoch 55
Validation binary_cross_entropy = 0.526863
Epoch 56
Validation binary_cross_entropy = 0.555936
Epoch 57
Validation binary_cross_entropy = 0.586559
Epoch 58
Validation binary_cross_entropy = 0.517623
Epoch 59
Loss = 1.1118e-01, PNorm = 99.0427, GNorm = 1.5593, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.518941
Epoch 60
Validation binary_cross_entropy = 0.475147
Epoch 61
Validation binary_cross_entropy = 0.503397
Epoch 62
Validation binary_cross_entropy = 0.499447
Epoch 63
Validation binary_cross_entropy = 0.510444
Epoch 64
Loss = 1.4828e-01, PNorm = 99.2888, GNorm = 2.6132, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.555212
Epoch 65
Validation binary_cross_entropy = 0.598210
Epoch 66
Validation binary_cross_entropy = 0.635819
Epoch 67
Validation binary_cross_entropy = 0.632163
Epoch 68
Validation binary_cross_entropy = 0.618202
Epoch 69
Loss = 9.4383e-02, PNorm = 99.5106, GNorm = 4.2381, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.629337
Epoch 70
Validation binary_cross_entropy = 0.602059
Epoch 71
Validation binary_cross_entropy = 0.599069
Epoch 72
Validation binary_cross_entropy = 0.597366
Epoch 73
Validation binary_cross_entropy = 0.570476
Epoch 74
Loss = 2.7518e-02, PNorm = 99.7012, GNorm = 0.9884, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.577324
Epoch 75
Validation binary_cross_entropy = 0.606908
Epoch 76
Validation binary_cross_entropy = 0.566526
Epoch 77
Validation binary_cross_entropy = 0.599820
Epoch 78
Validation binary_cross_entropy = 0.583991
Epoch 79
Loss = 2.2042e-02, PNorm = 99.8691, GNorm = 1.3618, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.601112
Epoch 80
Validation binary_cross_entropy = 0.634635
Epoch 81
Validation binary_cross_entropy = 0.725529
Epoch 82
Validation binary_cross_entropy = 0.663360
Epoch 83
Validation binary_cross_entropy = 0.650206
Epoch 84
Loss = 5.1628e-02, PNorm = 100.0498, GNorm = 1.9331, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.654119
Epoch 85
Validation binary_cross_entropy = 0.654891
Epoch 86
Validation binary_cross_entropy = 0.651248
Epoch 87
Validation binary_cross_entropy = 0.636543
Epoch 88
Validation binary_cross_entropy = 0.613442
Epoch 89
Loss = 2.2421e-02, PNorm = 100.2530, GNorm = 1.4355, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.620583
Epoch 90
Validation binary_cross_entropy = 0.634207
Epoch 91
Validation binary_cross_entropy = 0.643839
Epoch 92
Validation binary_cross_entropy = 0.656646
Epoch 93
Validation binary_cross_entropy = 0.668684
Epoch 94
Loss = 2.0036e-02, PNorm = 100.4785, GNorm = 1.0836, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.686735
Epoch 95
Validation binary_cross_entropy = 0.736710
Epoch 96
Validation binary_cross_entropy = 0.767143
Epoch 97
Validation binary_cross_entropy = 0.718871
Epoch 98
Validation binary_cross_entropy = 0.724248
Epoch 99
Loss = 8.9791e-03, PNorm = 100.6417, GNorm = 1.2028, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.729043
Epoch 100
Validation binary_cross_entropy = 0.726846
Epoch 101
Validation binary_cross_entropy = 0.803747
Epoch 102
Validation binary_cross_entropy = 0.717420
Epoch 103
Validation binary_cross_entropy = 0.754467
Epoch 104
Loss = 4.9458e-02, PNorm = 100.7758, GNorm = 2.4775, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.754944
Epoch 105
Validation binary_cross_entropy = 0.751741
Epoch 106
Validation binary_cross_entropy = 0.742936
Epoch 107
Validation binary_cross_entropy = 0.728850
Epoch 108
Validation binary_cross_entropy = 0.731412
Epoch 109
Loss = 3.2813e-02, PNorm = 100.9361, GNorm = 2.2794, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.749555
Epoch 110
Validation binary_cross_entropy = 0.865912
Epoch 111
Validation binary_cross_entropy = 0.691125
Epoch 112
Validation binary_cross_entropy = 0.695525
Epoch 113
Validation binary_cross_entropy = 0.712452
Epoch 114
Loss = 3.2770e-02, PNorm = 101.1164, GNorm = 3.9594, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.725704
Epoch 115
Validation binary_cross_entropy = 0.703999
Epoch 116
Validation binary_cross_entropy = 0.728321
Epoch 117
Validation binary_cross_entropy = 0.797982
Epoch 118
Validation binary_cross_entropy = 0.755984
Epoch 119
Loss = 3.5393e-02, PNorm = 101.3176, GNorm = 0.9244, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.767934
Epoch 120
Validation binary_cross_entropy = 0.740835
Epoch 121
Validation binary_cross_entropy = 0.743241
Epoch 122
Validation binary_cross_entropy = 0.757307
Epoch 123
Validation binary_cross_entropy = 0.796180
Epoch 124
Loss = 3.0687e-02, PNorm = 101.5122, GNorm = 2.0445, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.857679
Epoch 125
Validation binary_cross_entropy = 0.821245
Epoch 126
Validation binary_cross_entropy = 0.762899
Epoch 127
Validation binary_cross_entropy = 0.763730
Epoch 128
Validation binary_cross_entropy = 0.825498
Epoch 129
Loss = 4.7525e-02, PNorm = 101.7270, GNorm = 2.4410, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.817301
Epoch 130
Validation binary_cross_entropy = 0.774802
Epoch 131
Validation binary_cross_entropy = 0.768224
Epoch 132
Validation binary_cross_entropy = 0.764214
Epoch 133
Validation binary_cross_entropy = 0.765708
Epoch 134
Loss = 4.1977e-03, PNorm = 101.9196, GNorm = 0.2304, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.767226
Epoch 135
Validation binary_cross_entropy = 0.764492
Epoch 136
Validation binary_cross_entropy = 0.761535
Epoch 137
Validation binary_cross_entropy = 0.794875
Epoch 138
Validation binary_cross_entropy = 0.826240
Epoch 139
Loss = 2.0812e-02, PNorm = 102.0588, GNorm = 1.4131, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.902937
Epoch 140
Validation binary_cross_entropy = 0.942992
Epoch 141
Validation binary_cross_entropy = 0.933317
Epoch 142
Validation binary_cross_entropy = 0.915341
Epoch 143
Validation binary_cross_entropy = 0.950252
Epoch 144
Loss = 1.5609e-02, PNorm = 102.1864, GNorm = 0.1001, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.880271
Epoch 145
Validation binary_cross_entropy = 0.867568
Epoch 146
Validation binary_cross_entropy = 0.885731
Epoch 147
Validation binary_cross_entropy = 0.921293
Epoch 148
Validation binary_cross_entropy = 0.947309
Epoch 149
Loss = 1.7247e-02, PNorm = 102.2836, GNorm = 2.5611, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.926846
Epoch 150
Validation binary_cross_entropy = 0.908658
Epoch 151
Validation binary_cross_entropy = 0.912279
Epoch 152
Validation binary_cross_entropy = 0.925226
Epoch 153
Validation binary_cross_entropy = 0.944004
Epoch 154
Loss = 3.3395e-03, PNorm = 102.3652, GNorm = 0.2390, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.954763
Epoch 155
Validation binary_cross_entropy = 0.940320
Epoch 156
Validation binary_cross_entropy = 0.992256
Epoch 157
Validation binary_cross_entropy = 0.992352
Epoch 158
Validation binary_cross_entropy = 0.961716
Epoch 159
Loss = 1.1389e-02, PNorm = 102.4749, GNorm = 1.2237, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.918787
Epoch 160
Validation binary_cross_entropy = 0.912767
Epoch 161
Validation binary_cross_entropy = 0.901092
Epoch 162
Validation binary_cross_entropy = 0.904706
Epoch 163
Validation binary_cross_entropy = 0.964652
Epoch 164
Loss = 5.5430e-03, PNorm = 102.7054, GNorm = 1.0572, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.110426
Epoch 165
Validation binary_cross_entropy = 1.014816
Epoch 166
Validation binary_cross_entropy = 0.940114
Epoch 167
Validation binary_cross_entropy = 0.955668
Epoch 168
Validation binary_cross_entropy = 0.994256
Epoch 169
Loss = 6.7918e-03, PNorm = 102.9364, GNorm = 0.3744, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.047970
Epoch 170
Validation binary_cross_entropy = 1.018666
Epoch 171
Validation binary_cross_entropy = 1.051358
Epoch 172
Validation binary_cross_entropy = 1.280763
Epoch 173
Validation binary_cross_entropy = 1.303441
Epoch 174
Loss = 1.9901e-02, PNorm = 103.1437, GNorm = 2.7099, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.257182
Epoch 175
Validation binary_cross_entropy = 1.783089
Epoch 176
Validation binary_cross_entropy = 1.535671
Epoch 177
Validation binary_cross_entropy = 1.565416
Epoch 178
Validation binary_cross_entropy = 1.229473
Epoch 179
Loss = 1.5281e-01, PNorm = 103.3625, GNorm = 5.9456, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.117837
Epoch 180
Validation binary_cross_entropy = 0.993252
Epoch 181
Validation binary_cross_entropy = 2.032852
Epoch 182
Validation binary_cross_entropy = 0.932826
Epoch 183
Validation binary_cross_entropy = 0.951382
Epoch 184
Loss = 1.1564e-01, PNorm = 103.6690, GNorm = 0.9853, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.871113
Epoch 185
Validation binary_cross_entropy = 0.847279
Epoch 186
Validation binary_cross_entropy = 0.903414
Epoch 187
Validation binary_cross_entropy = 1.014082
Epoch 188
Validation binary_cross_entropy = 0.813766
Epoch 189
Loss = 6.1781e-03, PNorm = 103.9538, GNorm = 0.4207, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.301950
Epoch 190
Validation binary_cross_entropy = 0.755688
Epoch 191
Validation binary_cross_entropy = 0.778799
Epoch 192
Validation binary_cross_entropy = 0.846730
Epoch 193
Validation binary_cross_entropy = 0.843532
Epoch 194
Loss = 7.0043e-03, PNorm = 104.1728, GNorm = 0.3298, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.839552
Epoch 195
Validation binary_cross_entropy = 0.846426
Epoch 196
Validation binary_cross_entropy = 0.852841
Epoch 197
Validation binary_cross_entropy = 0.846808
Epoch 198
Validation binary_cross_entropy = 0.841329
Epoch 199
Loss = 2.8848e-02, PNorm = 104.3382, GNorm = 0.6471, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.859138
Epoch 200
Validation binary_cross_entropy = 0.854462
Epoch 201
Validation binary_cross_entropy = 0.837658
Epoch 202
Validation binary_cross_entropy = 0.848390
Epoch 203
Validation binary_cross_entropy = 0.871442
Epoch 204
Loss = 4.2001e-02, PNorm = 104.4971, GNorm = 0.6697, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.878747
Epoch 205
Validation binary_cross_entropy = 0.893914
Epoch 206
Validation binary_cross_entropy = 0.907347
Epoch 207
Validation binary_cross_entropy = 0.898523
Epoch 208
Validation binary_cross_entropy = 0.910844
Epoch 209
Loss = 9.6852e-03, PNorm = 104.6149, GNorm = 0.8406, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.915669
Epoch 210
Validation binary_cross_entropy = 0.914645
Epoch 211
Validation binary_cross_entropy = 0.916983
Epoch 212
Validation binary_cross_entropy = 0.922029
Epoch 213
Validation binary_cross_entropy = 0.941149
Epoch 214
Loss = 1.9411e-02, PNorm = 104.7075, GNorm = 1.9838, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.947859
Epoch 215
Validation binary_cross_entropy = 0.992564
Epoch 216
Validation binary_cross_entropy = 1.043008
Epoch 217
Validation binary_cross_entropy = 1.054018
Epoch 218
Validation binary_cross_entropy = 0.986881
Epoch 219
Loss = 1.5224e-03, PNorm = 104.7762, GNorm = 0.3089, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.959307
Epoch 220
Validation binary_cross_entropy = 0.952312
Epoch 221
Validation binary_cross_entropy = 0.956235
Epoch 222
Validation binary_cross_entropy = 0.978685
Epoch 223
Validation binary_cross_entropy = 1.012538
Epoch 224
Loss = 3.7954e-03, PNorm = 104.8468, GNorm = 0.1594, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.028530
Epoch 225
Validation binary_cross_entropy = 1.011874
Epoch 226
Validation binary_cross_entropy = 0.981456
Epoch 227
Validation binary_cross_entropy = 0.985080
Epoch 228
Validation binary_cross_entropy = 1.036045
Epoch 229
Loss = 3.6638e-03, PNorm = 104.9347, GNorm = 0.8213, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.114434
Epoch 230
Validation binary_cross_entropy = 1.085557
Epoch 231
Validation binary_cross_entropy = 1.065540
Epoch 232
Validation binary_cross_entropy = 1.053628
Epoch 233
Validation binary_cross_entropy = 1.049801
Epoch 234
Loss = 3.0665e-02, PNorm = 105.0232, GNorm = 3.3773, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.066757
Epoch 235
Validation binary_cross_entropy = 1.109005
Epoch 236
Validation binary_cross_entropy = 1.112082
Epoch 237
Validation binary_cross_entropy = 1.066276
Epoch 238
Validation binary_cross_entropy = 1.045521
Epoch 239
Loss = 4.5528e-03, PNorm = 105.0716, GNorm = 0.8608, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.040633
Epoch 240
Validation binary_cross_entropy = 1.041801
Epoch 241
Validation binary_cross_entropy = 1.050064
Epoch 242
Validation binary_cross_entropy = 1.096790
Epoch 243
Validation binary_cross_entropy = 1.140283
Epoch 244
Loss = 5.8586e-03, PNorm = 105.1217, GNorm = 1.2157, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.149299
Epoch 245
Validation binary_cross_entropy = 1.103342
Epoch 246
Validation binary_cross_entropy = 1.065774
Epoch 247
Validation binary_cross_entropy = 1.043886
Epoch 248
Validation binary_cross_entropy = 1.037563
Epoch 249
Loss = 2.7873e-04, PNorm = 105.1832, GNorm = 0.0311, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.031228
Epoch 250
Validation binary_cross_entropy = 1.046921
Epoch 251
Validation binary_cross_entropy = 1.102252
Epoch 252
Validation binary_cross_entropy = 1.170899
Epoch 253
Validation binary_cross_entropy = 1.188347
Epoch 254
Loss = 1.2322e-02, PNorm = 105.2404, GNorm = 1.9927, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.146938
Epoch 255
Validation binary_cross_entropy = 1.113758
Epoch 256
Validation binary_cross_entropy = 1.107902
Epoch 257
Validation binary_cross_entropy = 1.644363
Epoch 258
Validation binary_cross_entropy = 1.644353
Epoch 259
Loss = 4.4244e-04, PNorm = 105.3059, GNorm = 0.0227, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.642652
Epoch 260
Validation binary_cross_entropy = 1.110877
Epoch 261
Validation binary_cross_entropy = 1.125228
Epoch 262
Validation binary_cross_entropy = 1.103756
Epoch 263
Validation binary_cross_entropy = 1.074486
Epoch 264
Loss = 7.1115e-04, PNorm = 105.3566, GNorm = 0.0669, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.106859
Epoch 265
Validation binary_cross_entropy = 1.103670
Epoch 266
Validation binary_cross_entropy = 1.087882
Epoch 267
Validation binary_cross_entropy = 1.065660
Epoch 268
Validation binary_cross_entropy = 1.045465
Epoch 269
Loss = 4.6616e-04, PNorm = 105.4533, GNorm = 0.0635, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.033332
Epoch 270
Validation binary_cross_entropy = 1.027550
Epoch 271
Validation binary_cross_entropy = 1.025709
Epoch 272
Validation binary_cross_entropy = 1.026934
Epoch 273
Validation binary_cross_entropy = 1.030821
Epoch 274
Loss = 1.9554e-03, PNorm = 105.5193, GNorm = 0.2569, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.035968
Epoch 275
Validation binary_cross_entropy = 1.051729
Epoch 276
Validation binary_cross_entropy = 1.069554
Epoch 277
Validation binary_cross_entropy = 1.062582
Epoch 278
Validation binary_cross_entropy = 1.056033
Epoch 279
Loss = 2.5532e-03, PNorm = 105.5552, GNorm = 0.0166, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.061229
Epoch 280
Validation binary_cross_entropy = 1.113516
Epoch 281
Validation binary_cross_entropy = 1.159465
Epoch 282
Validation binary_cross_entropy = 1.631631
Epoch 283
Validation binary_cross_entropy = 1.597915
Epoch 284
Loss = 7.0848e-02, PNorm = 105.6365, GNorm = 0.2185, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.593791
Epoch 285
Validation binary_cross_entropy = 1.059595
Epoch 286
Validation binary_cross_entropy = 1.054473
Epoch 287
Validation binary_cross_entropy = 1.051954
Epoch 288
Validation binary_cross_entropy = 1.033337
Epoch 289
Loss = 1.7765e-03, PNorm = 105.7031, GNorm = 0.2042, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.009570
Epoch 290
Validation binary_cross_entropy = 0.997745
Epoch 291
Validation binary_cross_entropy = 0.991514
Epoch 292
Validation binary_cross_entropy = 0.996649
Epoch 293
Validation binary_cross_entropy = 1.017444
Epoch 294
Loss = 4.1103e-04, PNorm = 105.7636, GNorm = 0.0318, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.062787
Epoch 295
Validation binary_cross_entropy = 1.116408
Epoch 296
Validation binary_cross_entropy = 1.147492
Epoch 297
Validation binary_cross_entropy = 1.143131
Epoch 298
Validation binary_cross_entropy = 1.107813
Epoch 299
Loss = 5.3905e-03, PNorm = 105.8398, GNorm = 0.3872, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.055331
Model 0 best validation binary_cross_entropy = 0.288279 on epoch 15
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.187694
Ensemble test binary_cross_entropy = 0.187694
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.152941
Epoch 1
Validation binary_cross_entropy = 0.311554
Epoch 2
Validation binary_cross_entropy = 1.241249
Epoch 3
Validation binary_cross_entropy = 0.813296
Epoch 4
Loss = 6.4070e-01, PNorm = 96.4345, GNorm = 2.5744, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.267198
Epoch 5
Validation binary_cross_entropy = 1.218833
Epoch 6
Validation binary_cross_entropy = 1.439362
Epoch 7
Validation binary_cross_entropy = 0.293273
Epoch 8
Validation binary_cross_entropy = 0.440524
Epoch 9
Loss = 5.0247e-01, PNorm = 96.7705, GNorm = 5.1952, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 1.201356
Epoch 10
Validation binary_cross_entropy = 0.585677
Epoch 11
Validation binary_cross_entropy = 0.324653
Epoch 12
Validation binary_cross_entropy = 0.625810
Epoch 13
Validation binary_cross_entropy = 0.899161
Epoch 14
Loss = 3.6528e-01, PNorm = 97.1042, GNorm = 1.5068, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.418796
Epoch 15
Validation binary_cross_entropy = 0.364433
Epoch 16
Validation binary_cross_entropy = 0.694061
Epoch 17
Validation binary_cross_entropy = 0.529578
Epoch 18
Validation binary_cross_entropy = 0.344226
Epoch 19
Loss = 3.9565e-01, PNorm = 97.3501, GNorm = 4.0124, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.423127
Epoch 20
Validation binary_cross_entropy = 0.484277
Epoch 21
Validation binary_cross_entropy = 0.329391
Epoch 22
Validation binary_cross_entropy = 0.319454
Epoch 23
Validation binary_cross_entropy = 0.423849
Epoch 24
Loss = 1.8924e-01, PNorm = 97.5761, GNorm = 2.5446, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.399452
Epoch 25
Validation binary_cross_entropy = 0.350372
Epoch 26
Validation binary_cross_entropy = 0.408721
Epoch 27
Validation binary_cross_entropy = 0.389594
Epoch 28
Validation binary_cross_entropy = 0.384677
Epoch 29
Loss = 2.4434e-01, PNorm = 97.8030, GNorm = 3.1625, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.596957
Epoch 30
Validation binary_cross_entropy = 0.425685
Epoch 31
Validation binary_cross_entropy = 0.433564
Epoch 32
Validation binary_cross_entropy = 0.697525
Epoch 33
Validation binary_cross_entropy = 0.348633
Epoch 34
Loss = 4.0487e-01, PNorm = 98.0104, GNorm = 9.5693, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.330670
Epoch 35
Validation binary_cross_entropy = 0.592333
Epoch 36
Validation binary_cross_entropy = 0.446368
Epoch 37
Validation binary_cross_entropy = 0.367636
Epoch 38
Validation binary_cross_entropy = 0.372260
Epoch 39
Loss = 1.5004e-01, PNorm = 98.2346, GNorm = 1.6258, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.570480
Epoch 40
Validation binary_cross_entropy = 0.516257
Epoch 41
Validation binary_cross_entropy = 0.337020
Epoch 42
Validation binary_cross_entropy = 0.347195
Epoch 43
Validation binary_cross_entropy = 0.346136
Epoch 44
Loss = 1.6370e-01, PNorm = 98.4401, GNorm = 3.0243, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.377383
Epoch 45
Validation binary_cross_entropy = 0.383086
Epoch 46
Validation binary_cross_entropy = 0.371676
Epoch 47
Validation binary_cross_entropy = 0.372769
Epoch 48
Validation binary_cross_entropy = 0.413365
Epoch 49
Loss = 1.4228e-01, PNorm = 98.6452, GNorm = 0.7467, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.417119
Epoch 50
Validation binary_cross_entropy = 0.442533
Epoch 51
Validation binary_cross_entropy = 0.451397
Epoch 52
Validation binary_cross_entropy = 0.471931
Epoch 53
Validation binary_cross_entropy = 0.405330
Epoch 54
Loss = 1.2106e-01, PNorm = 98.8467, GNorm = 2.3742, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.407959
Epoch 55
Validation binary_cross_entropy = 0.418886
Epoch 56
Validation binary_cross_entropy = 0.442084
Epoch 57
Validation binary_cross_entropy = 0.436220
Epoch 58
Validation binary_cross_entropy = 0.428874
Epoch 59
Loss = 1.0126e-01, PNorm = 99.0755, GNorm = 2.0404, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.422249
Epoch 60
Validation binary_cross_entropy = 0.408959
Epoch 61
Validation binary_cross_entropy = 0.409298
Epoch 62
Validation binary_cross_entropy = 0.420320
Epoch 63
Validation binary_cross_entropy = 0.443458
Epoch 64
Loss = 3.5301e-02, PNorm = 99.2757, GNorm = 0.8917, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.465174
Epoch 65
Validation binary_cross_entropy = 0.515618
Epoch 66
Validation binary_cross_entropy = 0.510698
Epoch 67
Validation binary_cross_entropy = 0.541709
Epoch 68
Validation binary_cross_entropy = 0.536388
Epoch 69
Loss = 1.0127e-01, PNorm = 99.4727, GNorm = 2.4453, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.524993
Epoch 70
Validation binary_cross_entropy = 0.556250
Epoch 71
Validation binary_cross_entropy = 0.533978
Epoch 72
Validation binary_cross_entropy = 0.602021
Epoch 73
Validation binary_cross_entropy = 0.567955
Epoch 74
Loss = 1.0062e-01, PNorm = 99.6928, GNorm = 1.6192, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.521069
Epoch 75
Validation binary_cross_entropy = 0.517886
Epoch 76
Validation binary_cross_entropy = 0.654119
Epoch 77
Validation binary_cross_entropy = 0.442925
Epoch 78
Validation binary_cross_entropy = 0.513236
Epoch 79
Loss = 1.0315e-01, PNorm = 99.9288, GNorm = 4.2947, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.511777
Epoch 80
Validation binary_cross_entropy = 0.498504
Epoch 81
Validation binary_cross_entropy = 0.494247
Epoch 82
Validation binary_cross_entropy = 0.526455
Epoch 83
Validation binary_cross_entropy = 0.533627
Epoch 84
Loss = 1.1358e-01, PNorm = 100.1636, GNorm = 1.8207, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.530761
Epoch 85
Validation binary_cross_entropy = 0.539287
Epoch 86
Validation binary_cross_entropy = 0.547744
Epoch 87
Validation binary_cross_entropy = 0.572225
Epoch 88
Validation binary_cross_entropy = 0.614881
Epoch 89
Loss = 5.6809e-02, PNorm = 100.3852, GNorm = 2.1643, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.633877
Epoch 90
Validation binary_cross_entropy = 0.608894
Epoch 91
Validation binary_cross_entropy = 0.598246
Epoch 92
Validation binary_cross_entropy = 0.584127
Epoch 93
Validation binary_cross_entropy = 0.569030
Epoch 94
Loss = 4.2665e-02, PNorm = 100.6236, GNorm = 2.1674, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.537851
Epoch 95
Validation binary_cross_entropy = 0.618328
Epoch 96
Validation binary_cross_entropy = 0.682622
Epoch 97
Validation binary_cross_entropy = 0.659604
Epoch 98
Validation binary_cross_entropy = 0.706570
Epoch 99
Loss = 9.0992e-02, PNorm = 100.8805, GNorm = 4.2698, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.689821
Epoch 100
Validation binary_cross_entropy = 0.676527
Epoch 101
Validation binary_cross_entropy = 0.596036
Epoch 102
Validation binary_cross_entropy = 0.591451
Epoch 103
Validation binary_cross_entropy = 0.609020
Epoch 104
Loss = 3.0589e-02, PNorm = 101.0927, GNorm = 2.3309, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.624081
Epoch 105
Validation binary_cross_entropy = 0.633765
Epoch 106
Validation binary_cross_entropy = 0.683319
Epoch 107
Validation binary_cross_entropy = 0.738231
Epoch 108
Validation binary_cross_entropy = 0.683069
Epoch 109
Loss = 1.3314e-02, PNorm = 101.2598, GNorm = 0.4642, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.670511
Epoch 110
Validation binary_cross_entropy = 0.674831
Epoch 111
Validation binary_cross_entropy = 0.668167
Epoch 112
Validation binary_cross_entropy = 0.704636
Epoch 113
Validation binary_cross_entropy = 0.661365
Epoch 114
Loss = 3.1487e-02, PNorm = 101.4109, GNorm = 2.1040, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.634124
Epoch 115
Validation binary_cross_entropy = 0.651389
Epoch 116
Validation binary_cross_entropy = 0.703582
Epoch 117
Validation binary_cross_entropy = 0.770263
Epoch 118
Validation binary_cross_entropy = 0.707919
Epoch 119
Loss = 1.4389e-02, PNorm = 101.5643, GNorm = 0.3266, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.678164
Epoch 120
Validation binary_cross_entropy = 0.710345
Epoch 121
Validation binary_cross_entropy = 0.692729
Epoch 122
Validation binary_cross_entropy = 0.744879
Epoch 123
Validation binary_cross_entropy = 0.778140
Epoch 124
Loss = 2.8323e-02, PNorm = 101.7088, GNorm = 1.4332, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.754210
Epoch 125
Validation binary_cross_entropy = 0.730289
Epoch 126
Validation binary_cross_entropy = 0.746638
Epoch 127
Validation binary_cross_entropy = 0.796083
Epoch 128
Validation binary_cross_entropy = 0.850809
Epoch 129
Loss = 2.9023e-02, PNorm = 101.8322, GNorm = 0.8332, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.777847
Epoch 130
Validation binary_cross_entropy = 0.736144
Epoch 131
Validation binary_cross_entropy = 0.733865
Epoch 132
Validation binary_cross_entropy = 0.764585
Epoch 133
Validation binary_cross_entropy = 0.850735
Epoch 134
Loss = 5.2470e-02, PNorm = 101.9623, GNorm = 4.7376, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.745973
Epoch 135
Validation binary_cross_entropy = 0.771383
Epoch 136
Validation binary_cross_entropy = 0.807861
Epoch 137
Validation binary_cross_entropy = 0.927842
Epoch 138
Validation binary_cross_entropy = 0.860735
Epoch 139
Loss = 1.9342e-02, PNorm = 102.1489, GNorm = 1.0592, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.777503
Epoch 140
Validation binary_cross_entropy = 0.827187
Epoch 141
Validation binary_cross_entropy = 0.857098
Epoch 142
Validation binary_cross_entropy = 0.811696
Epoch 143
Validation binary_cross_entropy = 0.762440
Epoch 144
Loss = 1.2825e-02, PNorm = 102.3694, GNorm = 0.4581, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.683285
Epoch 145
Validation binary_cross_entropy = 0.723861
Epoch 146
Validation binary_cross_entropy = 0.727822
Epoch 147
Validation binary_cross_entropy = 0.709659
Epoch 148
Validation binary_cross_entropy = 0.699775
Epoch 149
Loss = 7.0257e-02, PNorm = 102.6001, GNorm = 1.0588, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.727783
Epoch 150
Validation binary_cross_entropy = 0.717743
Epoch 151
Validation binary_cross_entropy = 0.707840
Epoch 152
Validation binary_cross_entropy = 0.700916
Epoch 153
Validation binary_cross_entropy = 0.862606
Epoch 154
Loss = 6.4400e-02, PNorm = 102.8340, GNorm = 2.8563, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.755495
Epoch 155
Validation binary_cross_entropy = 0.650307
Epoch 156
Validation binary_cross_entropy = 0.715900
Epoch 157
Validation binary_cross_entropy = 0.668071
Epoch 158
Validation binary_cross_entropy = 0.796723
Epoch 159
Loss = 1.2146e-01, PNorm = 103.0500, GNorm = 4.2794, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.665635
Epoch 160
Validation binary_cross_entropy = 0.667566
Epoch 161
Validation binary_cross_entropy = 0.723036
Epoch 162
Validation binary_cross_entropy = 0.882676
Epoch 163
Validation binary_cross_entropy = 0.740628
Epoch 164
Loss = 5.5126e-02, PNorm = 103.2658, GNorm = 1.9473, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.858532
Epoch 165
Validation binary_cross_entropy = 0.760701
Epoch 166
Validation binary_cross_entropy = 0.690947
Epoch 167
Validation binary_cross_entropy = 0.696579
Epoch 168
Validation binary_cross_entropy = 0.717886
Epoch 169
Loss = 5.1974e-02, PNorm = 103.5154, GNorm = 2.5417, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.765520
Epoch 170
Validation binary_cross_entropy = 0.786996
Epoch 171
Validation binary_cross_entropy = 0.777121
Epoch 172
Validation binary_cross_entropy = 0.768226
Epoch 173
Validation binary_cross_entropy = 0.761970
Epoch 174
Loss = 7.3563e-03, PNorm = 103.7337, GNorm = 0.5089, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.767459
Epoch 175
Validation binary_cross_entropy = 0.778283
Epoch 176
Validation binary_cross_entropy = 0.777487
Epoch 177
Validation binary_cross_entropy = 0.792950
Epoch 178
Validation binary_cross_entropy = 0.800568
Epoch 179
Loss = 1.4323e-02, PNorm = 103.8803, GNorm = 1.0684, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.775053
Epoch 180
Validation binary_cross_entropy = 0.743164
Epoch 181
Validation binary_cross_entropy = 0.718369
Epoch 182
Validation binary_cross_entropy = 0.719411
Epoch 183
Validation binary_cross_entropy = 0.727009
Epoch 184
Loss = 2.9210e-02, PNorm = 103.9935, GNorm = 3.9438, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.758402
Epoch 185
Validation binary_cross_entropy = 0.860631
Epoch 186
Validation binary_cross_entropy = 0.861326
Epoch 187
Validation binary_cross_entropy = 0.763922
Epoch 188
Validation binary_cross_entropy = 0.734224
Epoch 189
Loss = 1.2221e-02, PNorm = 104.1048, GNorm = 2.1075, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.758852
Epoch 190
Validation binary_cross_entropy = 0.807099
Epoch 191
Validation binary_cross_entropy = 0.861201
Epoch 192
Validation binary_cross_entropy = 0.873090
Epoch 193
Validation binary_cross_entropy = 0.821963
Epoch 194
Loss = 2.8329e-03, PNorm = 104.2410, GNorm = 0.1371, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.784377
Epoch 195
Validation binary_cross_entropy = 0.761980
Epoch 196
Validation binary_cross_entropy = 0.759928
Epoch 197
Validation binary_cross_entropy = 0.748852
Epoch 198
Validation binary_cross_entropy = 0.746484
Epoch 199
Loss = 3.6330e-02, PNorm = 104.3627, GNorm = 4.1119, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.782308
Epoch 200
Validation binary_cross_entropy = 0.865918
Epoch 201
Validation binary_cross_entropy = 0.863467
Epoch 202
Validation binary_cross_entropy = 0.797194
Epoch 203
Validation binary_cross_entropy = 0.765136
Epoch 204
Loss = 1.0325e-02, PNorm = 104.4834, GNorm = 1.3149, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.785162
Epoch 205
Validation binary_cross_entropy = 0.811089
Epoch 206
Validation binary_cross_entropy = 0.805187
Epoch 207
Validation binary_cross_entropy = 0.866270
Epoch 208
Validation binary_cross_entropy = 0.936006
Epoch 209
Loss = 6.5785e-03, PNorm = 104.5731, GNorm = 0.3258, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.933571
Epoch 210
Validation binary_cross_entropy = 0.907556
Epoch 211
Validation binary_cross_entropy = 0.901047
Epoch 212
Validation binary_cross_entropy = 0.887943
Epoch 213
Validation binary_cross_entropy = 0.883985
Epoch 214
Loss = 2.3521e-03, PNorm = 104.6443, GNorm = 0.1091, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.883772
Epoch 215
Validation binary_cross_entropy = 0.885869
Epoch 216
Validation binary_cross_entropy = 0.886965
Epoch 217
Validation binary_cross_entropy = 0.889494
Epoch 218
Validation binary_cross_entropy = 0.893618
Epoch 219
Loss = 4.7625e-03, PNorm = 104.6917, GNorm = 0.0323, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.912479
Epoch 220
Validation binary_cross_entropy = 0.935898
Epoch 221
Validation binary_cross_entropy = 0.954803
Epoch 222
Validation binary_cross_entropy = 0.968624
Epoch 223
Validation binary_cross_entropy = 0.982511
Epoch 224
Loss = 3.0066e-03, PNorm = 104.7244, GNorm = 0.1607, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.991181
Epoch 225
Validation binary_cross_entropy = 0.993936
Epoch 226
Validation binary_cross_entropy = 0.986338
Epoch 227
Validation binary_cross_entropy = 0.979941
Epoch 228
Validation binary_cross_entropy = 0.976417
Epoch 229
Loss = 1.1811e-03, PNorm = 104.7492, GNorm = 0.0752, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.975207
Epoch 230
Validation binary_cross_entropy = 0.977753
Epoch 231
Validation binary_cross_entropy = 0.981330
Epoch 232
Validation binary_cross_entropy = 0.985135
Epoch 233
Validation binary_cross_entropy = 0.999262
Epoch 234
Loss = 3.5833e-04, PNorm = 104.7742, GNorm = 0.0174, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.026603
Epoch 235
Validation binary_cross_entropy = 1.056345
Epoch 236
Validation binary_cross_entropy = 1.080420
Epoch 237
Validation binary_cross_entropy = 1.054907
Epoch 238
Validation binary_cross_entropy = 1.028687
Epoch 239
Loss = 8.3621e-04, PNorm = 104.8089, GNorm = 0.0071, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.000408
Epoch 240
Validation binary_cross_entropy = 0.986674
Epoch 241
Validation binary_cross_entropy = 0.976945
Epoch 242
Validation binary_cross_entropy = 0.979122
Epoch 243
Validation binary_cross_entropy = 0.973871
Epoch 244
Loss = 5.1740e-03, PNorm = 104.8503, GNorm = 0.1500, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.995927
Epoch 245
Validation binary_cross_entropy = 1.014177
Epoch 246
Validation binary_cross_entropy = 1.032018
Epoch 247
Validation binary_cross_entropy = 1.037843
Epoch 248
Validation binary_cross_entropy = 1.037155
Epoch 249
Loss = 1.0978e-03, PNorm = 104.8850, GNorm = 0.0328, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.036165
Epoch 250
Validation binary_cross_entropy = 1.021472
Epoch 251
Validation binary_cross_entropy = 1.002813
Epoch 252
Validation binary_cross_entropy = 0.991575
Epoch 253
Validation binary_cross_entropy = 0.990289
Epoch 254
Loss = 2.3046e-04, PNorm = 104.9108, GNorm = 0.0036, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.994435
Epoch 255
Validation binary_cross_entropy = 1.003748
Epoch 256
Validation binary_cross_entropy = 1.053391
Epoch 257
Validation binary_cross_entropy = 1.115001
Epoch 258
Validation binary_cross_entropy = 1.154178
Epoch 259
Loss = 1.5484e-02, PNorm = 104.9310, GNorm = 0.3935, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.109566
Epoch 260
Validation binary_cross_entropy = 1.066150
Epoch 261
Validation binary_cross_entropy = 1.037623
Epoch 262
Validation binary_cross_entropy = 1.028376
Epoch 263
Validation binary_cross_entropy = 1.561245
Epoch 264
Loss = 6.7229e-03, PNorm = 104.9678, GNorm = 1.4826, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.576002
Epoch 265
Validation binary_cross_entropy = 1.069330
Epoch 266
Validation binary_cross_entropy = 1.094958
Epoch 267
Validation binary_cross_entropy = 1.116328
Epoch 268
Validation binary_cross_entropy = 1.134536
Epoch 269
Loss = 1.2823e-03, PNorm = 105.0009, GNorm = 0.2264, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.142673
Epoch 270
Validation binary_cross_entropy = 1.189962
Epoch 271
Validation binary_cross_entropy = 1.225871
Epoch 272
Validation binary_cross_entropy = 1.221277
Epoch 273
Validation binary_cross_entropy = 1.178741
Epoch 274
Loss = 1.1987e-03, PNorm = 105.0337, GNorm = 0.0275, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.139922
Epoch 275
Validation binary_cross_entropy = 1.111520
Epoch 276
Validation binary_cross_entropy = 1.090298
Epoch 277
Validation binary_cross_entropy = 1.078308
Epoch 278
Validation binary_cross_entropy = 1.075410
Epoch 279
Loss = 9.7888e-03, PNorm = 105.0576, GNorm = 0.0103, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.112565
Epoch 280
Validation binary_cross_entropy = 1.149905
Epoch 281
Validation binary_cross_entropy = 1.176540
Epoch 282
Validation binary_cross_entropy = 1.181888
Epoch 283
Validation binary_cross_entropy = 1.162766
Epoch 284
Loss = 7.3098e-04, PNorm = 105.0763, GNorm = 0.0575, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.132442
Epoch 285
Validation binary_cross_entropy = 1.110593
Epoch 286
Validation binary_cross_entropy = 1.097161
Epoch 287
Validation binary_cross_entropy = 1.114950
Epoch 288
Validation binary_cross_entropy = 1.131894
Epoch 289
Loss = 2.3360e-03, PNorm = 105.1194, GNorm = 0.5227, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.136266
Epoch 290
Validation binary_cross_entropy = 1.127340
Epoch 291
Validation binary_cross_entropy = 1.080508
Epoch 292
Validation binary_cross_entropy = 1.590025
Epoch 293
Validation binary_cross_entropy = 1.576321
Epoch 294
Loss = 6.4954e-04, PNorm = 105.1580, GNorm = 0.0337, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.570345
Epoch 295
Validation binary_cross_entropy = 1.567461
Epoch 296
Validation binary_cross_entropy = 1.566567
Epoch 297
Validation binary_cross_entropy = 1.583062
Epoch 298
Validation binary_cross_entropy = 1.074283
Epoch 299
Loss = 2.2708e-04, PNorm = 105.2032, GNorm = 0.0061, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.108170
Model 0 best validation binary_cross_entropy = 0.267198 on epoch 4
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.203045
Ensemble test binary_cross_entropy = 0.203045
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.487104
Epoch 1
Validation binary_cross_entropy = 0.299758
Epoch 2
Validation binary_cross_entropy = 1.102242
Epoch 3
Validation binary_cross_entropy = 0.928397
Epoch 4
Loss = 6.4638e-01, PNorm = 96.4340, GNorm = 4.6449, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.295617
Epoch 5
Validation binary_cross_entropy = 0.526496
Epoch 6
Validation binary_cross_entropy = 1.666461
Epoch 7
Validation binary_cross_entropy = 0.383772
Epoch 8
Validation binary_cross_entropy = 0.319450
Epoch 9
Loss = 4.8799e-01, PNorm = 96.7670, GNorm = 1.6254, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.986403
Epoch 10
Validation binary_cross_entropy = 0.982186
Epoch 11
Validation binary_cross_entropy = 0.285381
Epoch 12
Validation binary_cross_entropy = 0.399020
Epoch 13
Validation binary_cross_entropy = 0.585731
Epoch 14
Loss = 4.3043e-01, PNorm = 97.0980, GNorm = 1.0525, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.391788
Epoch 15
Validation binary_cross_entropy = 0.363447
Epoch 16
Validation binary_cross_entropy = 0.412353
Epoch 17
Validation binary_cross_entropy = 0.409589
Epoch 18
Validation binary_cross_entropy = 0.393366
Epoch 19
Loss = 2.4929e-01, PNorm = 97.3580, GNorm = 3.8307, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.411196
Epoch 20
Validation binary_cross_entropy = 0.356844
Epoch 21
Validation binary_cross_entropy = 0.705997
Epoch 22
Validation binary_cross_entropy = 0.359445
Epoch 23
Validation binary_cross_entropy = 0.419166
Epoch 24
Loss = 2.9029e-01, PNorm = 97.5791, GNorm = 2.7812, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.520443
Epoch 25
Validation binary_cross_entropy = 0.400603
Epoch 26
Validation binary_cross_entropy = 0.455637
Epoch 27
Validation binary_cross_entropy = 0.517509
Epoch 28
Validation binary_cross_entropy = 0.379390
Epoch 29
Loss = 2.9746e-01, PNorm = 97.7996, GNorm = 1.8003, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.405278
Epoch 30
Validation binary_cross_entropy = 0.429070
Epoch 31
Validation binary_cross_entropy = 0.359556
Epoch 32
Validation binary_cross_entropy = 0.410926
Epoch 33
Validation binary_cross_entropy = 0.360046
Epoch 34
Loss = 1.6822e-01, PNorm = 98.0280, GNorm = 2.5044, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.378525
Epoch 35
Validation binary_cross_entropy = 0.383470
Epoch 36
Validation binary_cross_entropy = 0.380172
Epoch 37
Validation binary_cross_entropy = 0.553213
Epoch 38
Validation binary_cross_entropy = 0.429288
Epoch 39
Loss = 2.2335e-01, PNorm = 98.2547, GNorm = 4.8831, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.424244
Epoch 40
Validation binary_cross_entropy = 0.445438
Epoch 41
Validation binary_cross_entropy = 0.430666
Epoch 42
Validation binary_cross_entropy = 0.421858
Epoch 43
Validation binary_cross_entropy = 0.439372
Epoch 44
Loss = 1.2231e-01, PNorm = 98.4683, GNorm = 1.9523, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.449700
Epoch 45
Validation binary_cross_entropy = 0.435886
Epoch 46
Validation binary_cross_entropy = 0.506027
Epoch 47
Validation binary_cross_entropy = 0.469968
Epoch 48
Validation binary_cross_entropy = 0.444369
Epoch 49
Loss = 8.5122e-02, PNorm = 98.6883, GNorm = 0.9852, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.558826
Epoch 50
Validation binary_cross_entropy = 0.509730
Epoch 51
Validation binary_cross_entropy = 0.602865
Epoch 52
Validation binary_cross_entropy = 0.565914
Epoch 53
Validation binary_cross_entropy = 0.483823
Epoch 54
Loss = 1.0933e-01, PNorm = 98.8951, GNorm = 3.9135, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.473609
Epoch 55
Validation binary_cross_entropy = 0.455201
Epoch 56
Validation binary_cross_entropy = 0.492699
Epoch 57
Validation binary_cross_entropy = 0.447097
Epoch 58
Validation binary_cross_entropy = 0.471028
Epoch 59
Loss = 9.9329e-02, PNorm = 99.1016, GNorm = 1.7115, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.458520
Epoch 60
Validation binary_cross_entropy = 0.472677
Epoch 61
Validation binary_cross_entropy = 0.474725
Epoch 62
Validation binary_cross_entropy = 0.464548
Epoch 63
Validation binary_cross_entropy = 0.480721
Epoch 64
Loss = 8.4537e-02, PNorm = 99.3167, GNorm = 2.7005, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.571626
Epoch 65
Validation binary_cross_entropy = 0.564471
Epoch 66
Validation binary_cross_entropy = 0.567724
Epoch 67
Validation binary_cross_entropy = 0.592164
Epoch 68
Validation binary_cross_entropy = 0.589748
Epoch 69
Loss = 1.0129e-01, PNorm = 99.5402, GNorm = 1.0469, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.589457
Epoch 70
Validation binary_cross_entropy = 0.601504
Epoch 71
Validation binary_cross_entropy = 0.612089
Epoch 72
Validation binary_cross_entropy = 0.580643
Epoch 73
Validation binary_cross_entropy = 0.601058
Epoch 74
Loss = 4.2716e-02, PNorm = 99.7620, GNorm = 1.6635, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.647954
Epoch 75
Validation binary_cross_entropy = 0.668539
Epoch 76
Validation binary_cross_entropy = 0.610958
Epoch 77
Validation binary_cross_entropy = 0.587500
Epoch 78
Validation binary_cross_entropy = 0.616698
Epoch 79
Loss = 1.8305e-01, PNorm = 99.9735, GNorm = 4.3332, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.606197
Epoch 80
Validation binary_cross_entropy = 0.563203
Epoch 81
Validation binary_cross_entropy = 0.560175
Epoch 82
Validation binary_cross_entropy = 0.582586
Epoch 83
Validation binary_cross_entropy = 0.572293
Epoch 84
Loss = 4.9307e-02, PNorm = 100.1715, GNorm = 2.8151, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.573214
Epoch 85
Validation binary_cross_entropy = 0.633232
Epoch 86
Validation binary_cross_entropy = 0.621772
Epoch 87
Validation binary_cross_entropy = 0.592371
Epoch 88
Validation binary_cross_entropy = 0.628018
Epoch 89
Loss = 6.3212e-02, PNorm = 100.3804, GNorm = 3.0597, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.746385
Epoch 90
Validation binary_cross_entropy = 0.681184
Epoch 91
Validation binary_cross_entropy = 0.708501
Epoch 92
Validation binary_cross_entropy = 0.755058
Epoch 93
Validation binary_cross_entropy = 0.718834
Epoch 94
Loss = 4.3080e-02, PNorm = 100.5953, GNorm = 2.0091, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.646684
Epoch 95
Validation binary_cross_entropy = 0.680103
Epoch 96
Validation binary_cross_entropy = 0.683283
Epoch 97
Validation binary_cross_entropy = 0.625150
Epoch 98
Validation binary_cross_entropy = 0.610789
Epoch 99
Loss = 4.0240e-02, PNorm = 100.8726, GNorm = 1.7602, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.606973
Epoch 100
Validation binary_cross_entropy = 0.709814
Epoch 101
Validation binary_cross_entropy = 0.744090
Epoch 102
Validation binary_cross_entropy = 0.695678
Epoch 103
Validation binary_cross_entropy = 0.690639
Epoch 104
Loss = 8.6088e-02, PNorm = 101.1186, GNorm = 3.2037, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.723704
Epoch 105
Validation binary_cross_entropy = 0.684460
Epoch 106
Validation binary_cross_entropy = 0.784616
Epoch 107
Validation binary_cross_entropy = 0.658357
Epoch 108
Validation binary_cross_entropy = 0.656238
Epoch 109
Loss = 3.0354e-02, PNorm = 101.3248, GNorm = 1.1499, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.635372
Epoch 110
Validation binary_cross_entropy = 0.675201
Epoch 111
Validation binary_cross_entropy = 0.786243
Epoch 112
Validation binary_cross_entropy = 0.708522
Epoch 113
Validation binary_cross_entropy = 0.699964
Epoch 114
Loss = 1.6527e-01, PNorm = 101.5298, GNorm = 1.6221, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.730267
Epoch 115
Validation binary_cross_entropy = 0.818720
Epoch 116
Validation binary_cross_entropy = 0.721319
Epoch 117
Validation binary_cross_entropy = 0.640297
Epoch 118
Validation binary_cross_entropy = 0.643672
Epoch 119
Loss = 6.8070e-02, PNorm = 101.7352, GNorm = 2.4476, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.621467
Epoch 120
Validation binary_cross_entropy = 0.692577
Epoch 121
Validation binary_cross_entropy = 0.763681
Epoch 122
Validation binary_cross_entropy = 0.713191
Epoch 123
Validation binary_cross_entropy = 0.666084
Epoch 124
Loss = 3.2693e-02, PNorm = 101.8932, GNorm = 1.7479, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.654207
Epoch 125
Validation binary_cross_entropy = 0.662336
Epoch 126
Validation binary_cross_entropy = 0.730538
Epoch 127
Validation binary_cross_entropy = 0.805031
Epoch 128
Validation binary_cross_entropy = 0.726168
Epoch 129
Loss = 1.5966e-02, PNorm = 102.0101, GNorm = 1.6169, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.685860
Epoch 130
Validation binary_cross_entropy = 0.716565
Epoch 131
Validation binary_cross_entropy = 0.780395
Epoch 132
Validation binary_cross_entropy = 0.844978
Epoch 133
Validation binary_cross_entropy = 0.819820
Epoch 134
Loss = 6.0217e-03, PNorm = 102.1365, GNorm = 0.3263, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.826678
Epoch 135
Validation binary_cross_entropy = 0.819394
Epoch 136
Validation binary_cross_entropy = 0.828658
Epoch 137
Validation binary_cross_entropy = 0.856537
Epoch 138
Validation binary_cross_entropy = 0.859369
Epoch 139
Loss = 9.0729e-03, PNorm = 102.2323, GNorm = 0.5715, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.815824
Epoch 140
Validation binary_cross_entropy = 0.789761
Epoch 141
Validation binary_cross_entropy = 0.777554
Epoch 142
Validation binary_cross_entropy = 0.792493
Epoch 143
Validation binary_cross_entropy = 0.852155
Epoch 144
Loss = 9.9986e-03, PNorm = 102.3047, GNorm = 0.5961, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.894144
Epoch 145
Validation binary_cross_entropy = 0.897826
Epoch 146
Validation binary_cross_entropy = 0.870199
Epoch 147
Validation binary_cross_entropy = 0.852483
Epoch 148
Validation binary_cross_entropy = 0.850569
Epoch 149
Loss = 2.1789e-02, PNorm = 102.3691, GNorm = 1.7178, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.867795
Epoch 150
Validation binary_cross_entropy = 0.904561
Epoch 151
Validation binary_cross_entropy = 0.952804
Epoch 152
Validation binary_cross_entropy = 0.933126
Epoch 153
Validation binary_cross_entropy = 0.859424
Epoch 154
Loss = 6.8692e-03, PNorm = 102.4482, GNorm = 0.6238, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.812415
Epoch 155
Validation binary_cross_entropy = 0.810875
Epoch 156
Validation binary_cross_entropy = 0.875066
Epoch 157
Validation binary_cross_entropy = 0.977514
Epoch 158
Validation binary_cross_entropy = 1.041697
Epoch 159
Loss = 2.4575e-02, PNorm = 102.5133, GNorm = 0.8762, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.946134
Epoch 160
Validation binary_cross_entropy = 0.878228
Epoch 161
Validation binary_cross_entropy = 0.855635
Epoch 162
Validation binary_cross_entropy = 0.858093
Epoch 163
Validation binary_cross_entropy = 0.905760
Epoch 164
Loss = 4.7273e-03, PNorm = 102.5966, GNorm = 0.2782, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.955687
Epoch 165
Validation binary_cross_entropy = 0.946935
Epoch 166
Validation binary_cross_entropy = 0.929000
Epoch 167
Validation binary_cross_entropy = 0.915912
Epoch 168
Validation binary_cross_entropy = 0.912969
Epoch 169
Loss = 1.3957e-02, PNorm = 102.6736, GNorm = 1.2192, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.921934
Epoch 170
Validation binary_cross_entropy = 0.928544
Epoch 171
Validation binary_cross_entropy = 0.931490
Epoch 172
Validation binary_cross_entropy = 0.946684
Epoch 173
Validation binary_cross_entropy = 1.003216
Epoch 174
Loss = 3.7986e-02, PNorm = 102.7438, GNorm = 1.9048, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.993409
Epoch 175
Validation binary_cross_entropy = 0.939077
Epoch 176
Validation binary_cross_entropy = 0.926062
Epoch 177
Validation binary_cross_entropy = 0.917301
Epoch 178
Validation binary_cross_entropy = 0.970401
Epoch 179
Loss = 7.5851e-03, PNorm = 102.8354, GNorm = 1.1045, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.057566
Epoch 180
Validation binary_cross_entropy = 1.026933
Epoch 181
Validation binary_cross_entropy = 0.920715
Epoch 182
Validation binary_cross_entropy = 0.876082
Epoch 183
Validation binary_cross_entropy = 1.410774
Epoch 184
Loss = 1.0743e-02, PNorm = 102.9710, GNorm = 1.1164, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.413408
Epoch 185
Validation binary_cross_entropy = 0.883084
Epoch 186
Validation binary_cross_entropy = 0.934039
Epoch 187
Validation binary_cross_entropy = 0.956580
Epoch 188
Validation binary_cross_entropy = 0.947496
Epoch 189
Loss = 6.2780e-03, PNorm = 103.1034, GNorm = 1.1813, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.928069
Epoch 190
Validation binary_cross_entropy = 0.918637
Epoch 191
Validation binary_cross_entropy = 0.916785
Epoch 192
Validation binary_cross_entropy = 0.901053
Epoch 193
Validation binary_cross_entropy = 0.919857
Epoch 194
Loss = 1.1089e-02, PNorm = 103.2156, GNorm = 0.1031, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.903457
Epoch 195
Validation binary_cross_entropy = 0.900957
Epoch 196
Validation binary_cross_entropy = 0.877705
Epoch 197
Validation binary_cross_entropy = 0.869100
Epoch 198
Validation binary_cross_entropy = 0.880476
Epoch 199
Loss = 1.8524e-03, PNorm = 103.3058, GNorm = 0.2408, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.904891
Epoch 200
Validation binary_cross_entropy = 0.934154
Epoch 201
Validation binary_cross_entropy = 0.954803
Epoch 202
Validation binary_cross_entropy = 0.960586
Epoch 203
Validation binary_cross_entropy = 0.965883
Epoch 204
Loss = 4.4125e-03, PNorm = 103.3773, GNorm = 0.1738, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.952876
Epoch 205
Validation binary_cross_entropy = 0.953968
Epoch 206
Validation binary_cross_entropy = 1.067731
Epoch 207
Validation binary_cross_entropy = 1.121831
Epoch 208
Validation binary_cross_entropy = 1.079322
Epoch 209
Loss = 7.9884e-03, PNorm = 103.4440, GNorm = 0.9534, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.993124
Epoch 210
Validation binary_cross_entropy = 0.940153
Epoch 211
Validation binary_cross_entropy = 0.931718
Epoch 212
Validation binary_cross_entropy = 0.940647
Epoch 213
Validation binary_cross_entropy = 0.971528
Epoch 214
Loss = 1.0897e-02, PNorm = 103.5132, GNorm = 1.3116, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.011707
Epoch 215
Validation binary_cross_entropy = 1.043033
Epoch 216
Validation binary_cross_entropy = 1.490675
Epoch 217
Validation binary_cross_entropy = 1.422363
Epoch 218
Validation binary_cross_entropy = 1.440159
Epoch 219
Loss = 3.5887e-02, PNorm = 103.6195, GNorm = 0.2245, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.074228
Epoch 220
Validation binary_cross_entropy = 1.239597
Epoch 221
Validation binary_cross_entropy = 1.021200
Epoch 222
Validation binary_cross_entropy = 1.425877
Epoch 223
Validation binary_cross_entropy = 1.460257
Epoch 224
Loss = 3.6281e-02, PNorm = 103.7634, GNorm = 2.3123, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.459003
Epoch 225
Validation binary_cross_entropy = 1.067331
Epoch 226
Validation binary_cross_entropy = 0.980699
Epoch 227
Validation binary_cross_entropy = 1.525587
Epoch 228
Validation binary_cross_entropy = 1.553713
Epoch 229
Loss = 6.5782e-02, PNorm = 103.9018, GNorm = 4.0202, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.824378
Epoch 230
Validation binary_cross_entropy = 0.927183
Epoch 231
Validation binary_cross_entropy = 0.906773
Epoch 232
Validation binary_cross_entropy = 0.841021
Epoch 233
Validation binary_cross_entropy = 0.884938
Epoch 234
Loss = 1.7989e-02, PNorm = 104.0753, GNorm = 2.4738, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.930516
Epoch 235
Validation binary_cross_entropy = 0.966489
Epoch 236
Validation binary_cross_entropy = 0.967593
Epoch 237
Validation binary_cross_entropy = 0.959947
Epoch 238
Validation binary_cross_entropy = 0.943561
Epoch 239
Loss = 1.4956e-02, PNorm = 104.2253, GNorm = 0.8760, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.926346
Epoch 240
Validation binary_cross_entropy = 0.918557
Epoch 241
Validation binary_cross_entropy = 0.916484
Epoch 242
Validation binary_cross_entropy = 0.956818
Epoch 243
Validation binary_cross_entropy = 0.995948
Epoch 244
Loss = 6.9269e-03, PNorm = 104.3260, GNorm = 0.1432, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.023586
Epoch 245
Validation binary_cross_entropy = 1.034895
Epoch 246
Validation binary_cross_entropy = 1.010254
Epoch 247
Validation binary_cross_entropy = 0.998794
Epoch 248
Validation binary_cross_entropy = 1.002222
Epoch 249
Loss = 3.0882e-03, PNorm = 104.4011, GNorm = 0.8420, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.009435
Epoch 250
Validation binary_cross_entropy = 1.020721
Epoch 251
Validation binary_cross_entropy = 1.037796
Epoch 252
Validation binary_cross_entropy = 1.066546
Epoch 253
Validation binary_cross_entropy = 1.099086
Epoch 254
Loss = 2.4250e-03, PNorm = 104.4638, GNorm = 0.3260, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.117809
Epoch 255
Validation binary_cross_entropy = 1.124384
Epoch 256
Validation binary_cross_entropy = 1.119798
Epoch 257
Validation binary_cross_entropy = 1.099396
Epoch 258
Validation binary_cross_entropy = 1.073747
Epoch 259
Loss = 8.2367e-04, PNorm = 104.5157, GNorm = 0.0615, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.051909
Epoch 260
Validation binary_cross_entropy = 1.034942
Epoch 261
Validation binary_cross_entropy = 1.039443
Epoch 262
Validation binary_cross_entropy = 1.088625
Epoch 263
Validation binary_cross_entropy = 1.121023
Epoch 264
Loss = 3.3738e-03, PNorm = 104.5638, GNorm = 0.0994, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.135892
Epoch 265
Validation binary_cross_entropy = 1.088653
Epoch 266
Validation binary_cross_entropy = 1.052905
Epoch 267
Validation binary_cross_entropy = 1.034007
Epoch 268
Validation binary_cross_entropy = 1.026583
Epoch 269
Loss = 2.9159e-04, PNorm = 104.6170, GNorm = 0.0267, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.021477
Epoch 270
Validation binary_cross_entropy = 1.037658
Epoch 271
Validation binary_cross_entropy = 1.045106
Epoch 272
Validation binary_cross_entropy = 1.055954
Epoch 273
Validation binary_cross_entropy = 1.066172
Epoch 274
Loss = 1.0620e-03, PNorm = 104.6530, GNorm = 0.0515, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.074366
Epoch 275
Validation binary_cross_entropy = 1.091420
Epoch 276
Validation binary_cross_entropy = 1.086442
Epoch 277
Validation binary_cross_entropy = 1.088979
Epoch 278
Validation binary_cross_entropy = 1.107424
Epoch 279
Loss = 1.2201e-03, PNorm = 104.7087, GNorm = 0.0855, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.124134
Epoch 280
Validation binary_cross_entropy = 1.129868
Epoch 281
Validation binary_cross_entropy = 1.112967
Epoch 282
Validation binary_cross_entropy = 1.597179
Epoch 283
Validation binary_cross_entropy = 1.565574
Epoch 284
Loss = 9.9642e-03, PNorm = 104.7572, GNorm = 2.3571, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.569874
Epoch 285
Validation binary_cross_entropy = 1.065909
Epoch 286
Validation binary_cross_entropy = 1.087100
Epoch 287
Validation binary_cross_entropy = 1.110527
Epoch 288
Validation binary_cross_entropy = 1.125379
Epoch 289
Loss = 2.8375e-03, PNorm = 104.7948, GNorm = 0.5713, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.137664
Epoch 290
Validation binary_cross_entropy = 1.174567
Epoch 291
Validation binary_cross_entropy = 1.202338
Epoch 292
Validation binary_cross_entropy = 1.183893
Epoch 293
Validation binary_cross_entropy = 1.142759
Epoch 294
Loss = 4.1102e-04, PNorm = 104.8417, GNorm = 0.0123, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.641394
Epoch 295
Validation binary_cross_entropy = 1.613846
Epoch 296
Validation binary_cross_entropy = 1.607051
Epoch 297
Validation binary_cross_entropy = 1.604387
Epoch 298
Validation binary_cross_entropy = 1.603679
Epoch 299
Loss = 9.5554e-04, PNorm = 104.8813, GNorm = 0.1753, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.605899
Model 0 best validation binary_cross_entropy = 0.285381 on epoch 11
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.203790
Ensemble test binary_cross_entropy = 0.203790
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2200, bias=False)
        (W_h): Linear(in_features=2200, out_features=2200, bias=False)
        (W_o): Linear(in_features=2333, out_features=2200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2500, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 20,644,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.486768
Epoch 1
Validation binary_cross_entropy = 0.301816
Epoch 2
Validation binary_cross_entropy = 1.197520
Epoch 3
Validation binary_cross_entropy = 1.527032
Epoch 4
Loss = 6.6274e-01, PNorm = 96.4334, GNorm = 2.4954, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.304888
Epoch 5
Validation binary_cross_entropy = 0.658711
Epoch 6
Validation binary_cross_entropy = 1.640114
Epoch 7
Validation binary_cross_entropy = 0.364494
Epoch 8
Validation binary_cross_entropy = 0.349789
Epoch 9
Loss = 4.8387e-01, PNorm = 96.7624, GNorm = 1.8994, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 1.058569
Epoch 10
Validation binary_cross_entropy = 0.756716
Epoch 11
Validation binary_cross_entropy = 0.322585
Epoch 12
Validation binary_cross_entropy = 0.428695
Epoch 13
Validation binary_cross_entropy = 0.901515
Epoch 14
Loss = 4.0809e-01, PNorm = 97.0866, GNorm = 3.6282, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.472619
Epoch 15
Validation binary_cross_entropy = 0.323439
Epoch 16
Validation binary_cross_entropy = 0.408524
Epoch 17
Validation binary_cross_entropy = 0.697286
Epoch 18
Validation binary_cross_entropy = 0.344022
Epoch 19
Loss = 2.9189e-01, PNorm = 97.3194, GNorm = 3.0087, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.364232
Epoch 20
Validation binary_cross_entropy = 0.449270
Epoch 21
Validation binary_cross_entropy = 0.362480
Epoch 22
Validation binary_cross_entropy = 0.349206
Epoch 23
Validation binary_cross_entropy = 0.413659
Epoch 24
Loss = 2.1617e-01, PNorm = 97.5262, GNorm = 1.5707, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.362147
Epoch 25
Validation binary_cross_entropy = 0.391009
Epoch 26
Validation binary_cross_entropy = 0.471108
Epoch 27
Validation binary_cross_entropy = 0.427809
Epoch 28
Validation binary_cross_entropy = 0.456020
Epoch 29
Loss = 1.7743e-01, PNorm = 97.7352, GNorm = 2.5104, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.436501
Epoch 30
Validation binary_cross_entropy = 0.385888
Epoch 31
Validation binary_cross_entropy = 0.420455
Epoch 32
Validation binary_cross_entropy = 0.450007
Epoch 33
Validation binary_cross_entropy = 0.902107
Epoch 34
Loss = 2.2752e-01, PNorm = 97.9640, GNorm = 5.3910, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.481897
Epoch 35
Validation binary_cross_entropy = 0.428989
Epoch 36
Validation binary_cross_entropy = 0.455815
Epoch 37
Validation binary_cross_entropy = 0.430909
Epoch 38
Validation binary_cross_entropy = 0.397890
Epoch 39
Loss = 1.8166e-01, PNorm = 98.2020, GNorm = 1.2899, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.730450
Epoch 40
Validation binary_cross_entropy = 0.461473
Epoch 41
Validation binary_cross_entropy = 0.579869
Epoch 42
Validation binary_cross_entropy = 0.467825
Epoch 43
Validation binary_cross_entropy = 0.437217
Epoch 44
Loss = 1.5371e-01, PNorm = 98.4483, GNorm = 2.4394, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.417289
Epoch 45
Validation binary_cross_entropy = 0.431728
Epoch 46
Validation binary_cross_entropy = 0.613786
Epoch 47
Validation binary_cross_entropy = 0.466224
Epoch 48
Validation binary_cross_entropy = 0.456481
Epoch 49
Loss = 1.0296e-01, PNorm = 98.7039, GNorm = 0.9841, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.445723
Epoch 50
Validation binary_cross_entropy = 0.433434
Epoch 51
Validation binary_cross_entropy = 0.437257
Epoch 52
Validation binary_cross_entropy = 0.501936
Epoch 53
Validation binary_cross_entropy = 0.502328
Epoch 54
Loss = 6.6148e-02, PNorm = 98.9167, GNorm = 1.5384, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.453455
Epoch 55
Validation binary_cross_entropy = 0.466449
Epoch 56
Validation binary_cross_entropy = 0.472738
Epoch 57
Validation binary_cross_entropy = 0.497781
Epoch 58
Validation binary_cross_entropy = 0.505093
Epoch 59
Loss = 6.6081e-02, PNorm = 99.1170, GNorm = 3.0237, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.560811
Epoch 60
Validation binary_cross_entropy = 0.622682
Epoch 61
Validation binary_cross_entropy = 0.594724
Epoch 62
Validation binary_cross_entropy = 0.580934
Epoch 63
Validation binary_cross_entropy = 0.599890
Epoch 64
Loss = 7.2648e-02, PNorm = 99.3064, GNorm = 2.7818, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.609457
Epoch 65
Validation binary_cross_entropy = 0.600757
Epoch 66
Validation binary_cross_entropy = 0.590189
Epoch 67
Validation binary_cross_entropy = 0.603149
Epoch 68
Validation binary_cross_entropy = 0.613573
Epoch 69
Loss = 4.2185e-02, PNorm = 99.5154, GNorm = 2.8810, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.656981
Epoch 70
Validation binary_cross_entropy = 0.673884
Epoch 71
Validation binary_cross_entropy = 0.739342
Epoch 72
Validation binary_cross_entropy = 0.661331
Epoch 73
Validation binary_cross_entropy = 0.674959
Epoch 74
Loss = 1.5141e-01, PNorm = 99.6884, GNorm = 3.8039, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.752753
Epoch 75
Validation binary_cross_entropy = 0.689326
Epoch 76
Validation binary_cross_entropy = 0.762626
Epoch 77
Validation binary_cross_entropy = 0.578698
Epoch 78
Validation binary_cross_entropy = 0.688438
Epoch 79
Loss = 1.0276e-01, PNorm = 99.8925, GNorm = 2.4836, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.585416
Epoch 80
Validation binary_cross_entropy = 0.556160
Epoch 81
Validation binary_cross_entropy = 0.719278
Epoch 82
Validation binary_cross_entropy = 0.763374
Epoch 83
Validation binary_cross_entropy = 0.679339
Epoch 84
Loss = 2.0079e-01, PNorm = 100.1142, GNorm = 8.1005, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.558262
Epoch 85
Validation binary_cross_entropy = 0.525001
Epoch 86
Validation binary_cross_entropy = 0.574147
Epoch 87
Validation binary_cross_entropy = 0.530899
Epoch 88
Validation binary_cross_entropy = 0.554273
Epoch 89
Loss = 9.6293e-02, PNorm = 100.3416, GNorm = 2.5108, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.527761
Epoch 90
Validation binary_cross_entropy = 0.758878
Epoch 91
Validation binary_cross_entropy = 0.525471
Epoch 92
Validation binary_cross_entropy = 0.604939
Epoch 93
Validation binary_cross_entropy = 0.560979
Epoch 94
Loss = 2.7605e-02, PNorm = 100.6417, GNorm = 0.7466, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.496869
Epoch 95
Validation binary_cross_entropy = 0.555455
Epoch 96
Validation binary_cross_entropy = 0.559444
Epoch 97
Validation binary_cross_entropy = 0.552388
Epoch 98
Validation binary_cross_entropy = 0.570049
Epoch 99
Loss = 4.6281e-02, PNorm = 100.9641, GNorm = 2.1046, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.562407
Epoch 100
Validation binary_cross_entropy = 0.554110
Epoch 101
Validation binary_cross_entropy = 0.562681
Epoch 102
Validation binary_cross_entropy = 0.610044
Epoch 103
Validation binary_cross_entropy = 0.599579
Epoch 104
Loss = 1.1044e-02, PNorm = 101.1898, GNorm = 0.1737, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.567303
Epoch 105
Validation binary_cross_entropy = 0.566895
Epoch 106
Validation binary_cross_entropy = 0.618114
Epoch 107
Validation binary_cross_entropy = 0.653602
Epoch 108
Validation binary_cross_entropy = 0.673619
Epoch 109
Loss = 3.3056e-02, PNorm = 101.3779, GNorm = 2.2545, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.680490
Epoch 110
Validation binary_cross_entropy = 0.714316
Epoch 111
Validation binary_cross_entropy = 0.728838
Epoch 112
Validation binary_cross_entropy = 0.719203
Epoch 113
Validation binary_cross_entropy = 0.731649
Epoch 114
Loss = 5.2710e-02, PNorm = 101.5339, GNorm = 3.7018, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.772601
Epoch 115
Validation binary_cross_entropy = 0.809368
Epoch 116
Validation binary_cross_entropy = 0.747613
Epoch 117
Validation binary_cross_entropy = 0.689166
Epoch 118
Validation binary_cross_entropy = 0.693430
Epoch 119
Loss = 1.3470e-02, PNorm = 101.6594, GNorm = 0.7222, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.708514
Epoch 120
Validation binary_cross_entropy = 0.745036
Epoch 121
Validation binary_cross_entropy = 0.774722
Epoch 122
Validation binary_cross_entropy = 0.761119
Epoch 123
Validation binary_cross_entropy = 0.739044
Epoch 124
Loss = 2.2718e-02, PNorm = 101.7803, GNorm = 2.3998, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.748999
Epoch 125
Validation binary_cross_entropy = 0.767390
Epoch 126
Validation binary_cross_entropy = 0.801604
Epoch 127
Validation binary_cross_entropy = 0.840704
Epoch 128
Validation binary_cross_entropy = 0.850160
Epoch 129
Loss = 6.1552e-03, PNorm = 101.8964, GNorm = 0.6750, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.829465
Epoch 130
Validation binary_cross_entropy = 0.826611
Epoch 131
Validation binary_cross_entropy = 0.843969
Epoch 132
Validation binary_cross_entropy = 0.896715
Epoch 133
Validation binary_cross_entropy = 0.897692
Epoch 134
Loss = 1.3958e-02, PNorm = 102.0017, GNorm = 1.4911, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.867776
Epoch 135
Validation binary_cross_entropy = 0.887828
Epoch 136
Validation binary_cross_entropy = 0.888123
Epoch 137
Validation binary_cross_entropy = 0.863147
Epoch 138
Validation binary_cross_entropy = 0.791072
Epoch 139
Loss = 2.8379e-02, PNorm = 102.1049, GNorm = 2.0049, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.812905
Epoch 140
Validation binary_cross_entropy = 0.887433
Epoch 141
Validation binary_cross_entropy = 0.924564
Epoch 142
Validation binary_cross_entropy = 0.868050
Epoch 143
Validation binary_cross_entropy = 0.831248
Epoch 144
Loss = 5.3328e-02, PNorm = 102.2385, GNorm = 2.9406, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.830139
Epoch 145
Validation binary_cross_entropy = 0.865601
Epoch 146
Validation binary_cross_entropy = 0.804044
Epoch 147
Validation binary_cross_entropy = 0.748287
Epoch 148
Validation binary_cross_entropy = 0.761841
Epoch 149
Loss = 4.2048e-02, PNorm = 102.3762, GNorm = 2.4857, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.859178
Epoch 150
Validation binary_cross_entropy = 0.902250
Epoch 151
Validation binary_cross_entropy = 0.830522
Epoch 152
Validation binary_cross_entropy = 0.804775
Epoch 153
Validation binary_cross_entropy = 0.807011
Epoch 154
Loss = 4.8219e-02, PNorm = 102.4905, GNorm = 0.2835, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.803990
Epoch 155
Validation binary_cross_entropy = 0.870790
Epoch 156
Validation binary_cross_entropy = 0.844787
Epoch 157
Validation binary_cross_entropy = 0.793058
Epoch 158
Validation binary_cross_entropy = 0.771996
Epoch 159
Loss = 2.0977e-02, PNorm = 102.6273, GNorm = 1.8256, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.779670
Epoch 160
Validation binary_cross_entropy = 0.787929
Epoch 161
Validation binary_cross_entropy = 0.756691
Epoch 162
Validation binary_cross_entropy = 0.764979
Epoch 163
Validation binary_cross_entropy = 0.799818
Epoch 164
Loss = 1.0473e-02, PNorm = 102.7903, GNorm = 0.6140, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.842340
Epoch 165
Validation binary_cross_entropy = 0.860801
Epoch 166
Validation binary_cross_entropy = 0.836154
Epoch 167
Validation binary_cross_entropy = 0.832679
Epoch 168
Validation binary_cross_entropy = 0.868279
Epoch 169
Loss = 8.0949e-03, PNorm = 102.9200, GNorm = 0.9683, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.922811
Epoch 170
Validation binary_cross_entropy = 0.916796
Epoch 171
Validation binary_cross_entropy = 0.884055
Epoch 172
Validation binary_cross_entropy = 0.891362
Epoch 173
Validation binary_cross_entropy = 0.901583
Epoch 174
Loss = 6.9857e-03, PNorm = 103.0671, GNorm = 0.5027, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.906714
Epoch 175
Validation binary_cross_entropy = 0.900897
Epoch 176
Validation binary_cross_entropy = 0.884232
Epoch 177
Validation binary_cross_entropy = 0.872809
Epoch 178
Validation binary_cross_entropy = 0.864393
Epoch 179
Loss = 1.5493e-02, PNorm = 103.1794, GNorm = 0.8825, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.891128
Epoch 180
Validation binary_cross_entropy = 0.875176
Epoch 181
Validation binary_cross_entropy = 0.848586
Epoch 182
Validation binary_cross_entropy = 0.851362
Epoch 183
Validation binary_cross_entropy = 0.865433
Epoch 184
Loss = 5.6960e-03, PNorm = 103.2883, GNorm = 0.8544, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.880554
Epoch 185
Validation binary_cross_entropy = 0.985054
Epoch 186
Validation binary_cross_entropy = 0.839625
Epoch 187
Validation binary_cross_entropy = 0.824349
Epoch 188
Validation binary_cross_entropy = 0.839440
Epoch 189
Loss = 3.2050e-03, PNorm = 103.4478, GNorm = 0.3868, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.865156
Epoch 190
Validation binary_cross_entropy = 0.888234
Epoch 191
Validation binary_cross_entropy = 0.877691
Epoch 192
Validation binary_cross_entropy = 0.869842
Epoch 193
Validation binary_cross_entropy = 0.867450
Epoch 194
Loss = 8.3940e-03, PNorm = 103.5934, GNorm = 0.0429, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.849595
Epoch 195
Validation binary_cross_entropy = 0.865599
Epoch 196
Validation binary_cross_entropy = 0.961921
Epoch 197
Validation binary_cross_entropy = 0.926623
Epoch 198
Validation binary_cross_entropy = 0.849394
Epoch 199
Loss = 2.4494e-03, PNorm = 103.7478, GNorm = 0.3444, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.823919
Epoch 200
Validation binary_cross_entropy = 0.821443
Epoch 201
Validation binary_cross_entropy = 0.824310
Epoch 202
Validation binary_cross_entropy = 0.857573
Epoch 203
Validation binary_cross_entropy = 0.925750
Epoch 204
Loss = 1.0916e-02, PNorm = 103.8752, GNorm = 1.3741, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.954702
Epoch 205
Validation binary_cross_entropy = 0.854159
Epoch 206
Validation binary_cross_entropy = 0.814271
Epoch 207
Validation binary_cross_entropy = 0.807035
Epoch 208
Validation binary_cross_entropy = 0.793888
Epoch 209
Loss = 6.2473e-03, PNorm = 103.9954, GNorm = 0.1787, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.814610
Epoch 210
Validation binary_cross_entropy = 0.864107
Epoch 211
Validation binary_cross_entropy = 0.878645
Epoch 212
Validation binary_cross_entropy = 0.861162
Epoch 213
Validation binary_cross_entropy = 0.807773
Epoch 214
Loss = 1.8788e-02, PNorm = 104.0666, GNorm = 0.1251, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.793033
Epoch 215
Validation binary_cross_entropy = 0.793065
Epoch 216
Validation binary_cross_entropy = 0.800156
Epoch 217
Validation binary_cross_entropy = 0.809716
Epoch 218
Validation binary_cross_entropy = 0.826800
Epoch 219
Loss = 1.0391e-03, PNorm = 104.1183, GNorm = 0.0245, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.845226
Epoch 220
Validation binary_cross_entropy = 0.864081
Epoch 221
Validation binary_cross_entropy = 0.886162
Epoch 222
Validation binary_cross_entropy = 0.904536
Epoch 223
Validation binary_cross_entropy = 0.914052
Epoch 224
Loss = 1.8619e-03, PNorm = 104.1682, GNorm = 0.1568, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.916104
Epoch 225
Validation binary_cross_entropy = 0.895208
Epoch 226
Validation binary_cross_entropy = 0.881823
Epoch 227
Validation binary_cross_entropy = 0.892609
Epoch 228
Validation binary_cross_entropy = 0.911097
Epoch 229
Loss = 2.7006e-03, PNorm = 104.1986, GNorm = 0.0979, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.927220
Epoch 230
Validation binary_cross_entropy = 0.930931
Epoch 231
Validation binary_cross_entropy = 0.914546
Epoch 232
Validation binary_cross_entropy = 0.946277
Epoch 233
Validation binary_cross_entropy = 0.982121
Epoch 234
Loss = 8.2904e-03, PNorm = 104.2544, GNorm = 1.6668, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.997713
Epoch 235
Validation binary_cross_entropy = 0.968826
Epoch 236
Validation binary_cross_entropy = 0.941150
Epoch 237
Validation binary_cross_entropy = 0.915357
Epoch 238
Validation binary_cross_entropy = 0.896852
Epoch 239
Loss = 2.9930e-04, PNorm = 104.3076, GNorm = 0.0179, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.895032
Epoch 240
Validation binary_cross_entropy = 0.895379
Epoch 241
Validation binary_cross_entropy = 0.890938
Epoch 242
Validation binary_cross_entropy = 0.945569
Epoch 243
Validation binary_cross_entropy = 0.987484
Epoch 244
Loss = 1.4376e-02, PNorm = 104.3668, GNorm = 2.5131, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.967361
Epoch 245
Validation binary_cross_entropy = 0.927820
Epoch 246
Validation binary_cross_entropy = 0.900671
Epoch 247
Validation binary_cross_entropy = 0.887436
Epoch 248
Validation binary_cross_entropy = 0.884988
Epoch 249
Loss = 1.1895e-02, PNorm = 104.4229, GNorm = 0.0594, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.902044
Epoch 250
Validation binary_cross_entropy = 0.942030
Epoch 251
Validation binary_cross_entropy = 0.985953
Epoch 252
Validation binary_cross_entropy = 1.013704
Epoch 253
Validation binary_cross_entropy = 1.026242
Epoch 254
Loss = 6.5294e-03, PNorm = 104.4757, GNorm = 0.8315, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.001109
Epoch 255
Validation binary_cross_entropy = 0.971703
Epoch 256
Validation binary_cross_entropy = 0.948151
Epoch 257
Validation binary_cross_entropy = 0.927681
Epoch 258
Validation binary_cross_entropy = 0.913872
Epoch 259
Loss = 3.7188e-03, PNorm = 104.5289, GNorm = 0.0217, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.919228
Epoch 260
Validation binary_cross_entropy = 0.919503
Epoch 261
Validation binary_cross_entropy = 0.921683
Epoch 262
Validation binary_cross_entropy = 0.918425
Epoch 263
Validation binary_cross_entropy = 0.916373
Epoch 264
Loss = 6.2103e-04, PNorm = 104.5693, GNorm = 0.0250, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.922608
Epoch 265
Validation binary_cross_entropy = 0.928088
Epoch 266
Validation binary_cross_entropy = 0.936217
Epoch 267
Validation binary_cross_entropy = 0.951287
Epoch 268
Validation binary_cross_entropy = 0.962718
Epoch 269
Loss = 2.2962e-03, PNorm = 104.5973, GNorm = 0.0205, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.978954
Epoch 270
Validation binary_cross_entropy = 0.984277
Epoch 271
Validation binary_cross_entropy = 0.994813
Epoch 272
Validation binary_cross_entropy = 1.004622
Epoch 273
Validation binary_cross_entropy = 1.013111
Epoch 274
Loss = 8.1415e-03, PNorm = 104.6231, GNorm = 0.0245, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.037913
Epoch 275
Validation binary_cross_entropy = 1.054272
Epoch 276
Validation binary_cross_entropy = 1.071912
Epoch 277
Validation binary_cross_entropy = 1.081058
Epoch 278
Validation binary_cross_entropy = 1.080732
Epoch 279
Loss = 1.0453e-03, PNorm = 104.6593, GNorm = 0.0284, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.075972
Epoch 280
Validation binary_cross_entropy = 1.064503
Epoch 281
Validation binary_cross_entropy = 1.057082
Epoch 282
Validation binary_cross_entropy = 1.046229
Epoch 283
Validation binary_cross_entropy = 1.041136
Epoch 284
Loss = 1.1153e-03, PNorm = 104.6870, GNorm = 0.2494, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.039321
Epoch 285
Validation binary_cross_entropy = 1.039454
Epoch 286
Validation binary_cross_entropy = 1.042194
Epoch 287
Validation binary_cross_entropy = 1.047161
Epoch 288
Validation binary_cross_entropy = 1.050919
Epoch 289
Loss = 2.5773e-04, PNorm = 104.7012, GNorm = 0.0217, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.053979
Epoch 290
Validation binary_cross_entropy = 1.056023
Epoch 291
Validation binary_cross_entropy = 1.055646
Epoch 292
Validation binary_cross_entropy = 1.055804
Epoch 293
Validation binary_cross_entropy = 1.055912
Epoch 294
Loss = 5.2383e-04, PNorm = 104.7092, GNorm = 0.1035, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.057137
Epoch 295
Validation binary_cross_entropy = 1.059125
Epoch 296
Validation binary_cross_entropy = 1.059876
Epoch 297
Validation binary_cross_entropy = 1.060659
Epoch 298
Validation binary_cross_entropy = 1.053148
Epoch 299
Loss = 1.1466e-04, PNorm = 104.7125, GNorm = 0.0118, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.047554
Model 0 best validation binary_cross_entropy = 0.301816 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.200732
Ensemble test binary_cross_entropy = 0.200732
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.200234
	Seed 1 ==> test binary_cross_entropy = 0.221389
	Seed 2 ==> test binary_cross_entropy = 0.198511
	Seed 3 ==> test binary_cross_entropy = 0.195845
	Seed 4 ==> test binary_cross_entropy = 0.197648
	Seed 5 ==> test binary_cross_entropy = 0.220936
	Seed 6 ==> test binary_cross_entropy = 0.187694
	Seed 7 ==> test binary_cross_entropy = 0.203045
	Seed 8 ==> test binary_cross_entropy = 0.203790
	Seed 9 ==> test binary_cross_entropy = 0.200732
Overall test binary_cross_entropy = 0.202982 +/- 0.010038
Elapsed time = 0:08:06
